If you would like to contact us, please fill in the form below. We aim to respond to email enquiries as quickly as possible but please allow two working days for a reply.
Contact Us
Required value*
With details email us: testerblog00@gmail.com
Why this post?
The foundation for robust GUI automated checks is writing good element locators. Xpath is one locator strategy used for selecting nodes from Document Object Models (DOM) like XML, HTML, etc. Generating XPaths manually is a routine and time-consuming task.
As part of simplifying our test writing process, we came up with a thought to write a utility script which will identify and auto-generate robust and simple XPaths. We started off with generating XPaths for Input and Button fields of a webpage using the general locators like id, name, class etc. The Python program in this post will demonstrate how to create XPath expressions automatically for input and button tags. Whenever there is a requirement to automate a webpage, the tester can simply run this script which would generate a bunch of XPaths without any human intervention.
What should be considered when writing an XPath?
Most of us rely on extracting the XPaths from browser plugins or tools. These tools have got limitations. One of the limitations is that they frequently produce absolute XPaths (meaning all the way from the HTML tag!) which are long and extremely flaky. Other smarter tools seem to have trouble producing unique XPaths in anything but the simplest conditions. Generally, we locate the elements using their unique attributes but some elements do not have unique attributes. Locating such elements is difficult because the XPath generated will have multiple matching elements. It is important to consider the following points while choosing an XPath. A good locator is:
- Unique – XPath should have only one candidate element (Unique).
- Descriptive – It is easy to identify the element easily when the XPath is descriptive.
- Shorter in length – You will have multiple XPath options. A shorter XPath shall be selected to make it more readable.
- Valid even after changes to a page – XPath should be selected in such a way it is valid even after changes in DOM.
Overview of our Python Utility script
We made an effort to write a script which will auto-generate XPaths for Input and Button elements in the webpage and also check for the uniqueness of the generated XPath. This will save the automation time and effort. In the coming sections, we will be talking about below items:
- Accept a URL and parse the page content using BeautifulSoup
- For each element, check for the existence of the attribute and guess the XPath
- Check for uniqueness of the generated XPath
- How we tested this utility
- Putting it all together
Note:- If the XPath generated is not unique or if the HTML page does not have the attribute mentioned for the given tag then our script does not generate any XPaths.
Accept a URL and Parse the page content using BeautifulSoup
First, we need to import all the libraries that we are going to use. We used python module BeautifulSoup to extract and parse the HTML content.
Our main method starts with declaring a variable which accepts the input URL of the page.
The next step is to parse the page into a BeautifulSoup format. We used selenium’s execute_script() function to get the inner HTML of a page and return it as a string to the Python script. This method takes as a parameter a string of Javascript code that it executes inside of the browser.
For each element, check for existence of the attribute and guess the XPath
Now we have a variable, soup, containing the HTML of the page. Here’s where we can start coding the part that extracts the data. BeautifulSoup can help us get into these layers and extract the content with find_all() method. Using this method we are going to fetch all the Input and Button tags from the HTML page. We are passing the ‘soup’ as an argument for generate_xpath method.
If the webpage doesn’t have any inputs and buttons it throws a print message saying that there are no tags to generate the Xpaths for this URL.
Now let us look into the generate_xpath() logic. We first initialized few list variables for element lists and attribute lists. guessable_elements lists consists of element lists and known_attribute_list consists of attribute lists. We are looping over element lists (as of now we are doing this only for input and button elements). Next for each element we will loop over the attribute lists and check for the attribute existence and then guess the XPath. Please note that we have declared the attribute lists based on the order of importance of attribute occurrence. For eg:- If ‘id‘ is not available for that element, next we are checking for ‘name‘ attribute and so on.
def __init__(self):
self.elements = None
self.guessable_elements = ['input','button','img','link','text','div']
self.known_attribute_list = ['id','name','placeholder','value','title',
'type','class','href','alt','img','text','link','real','property'] def generate_xpath(self,soup): "generate the xpath" result_flag = False try: for guessable_element in self.guessable_elements: #Print tag name self.elements = soup.find_all(guessable_element) #DoM tag of guessable_element for element in self.elements: if (not element.has_attr("type")) or (element.has_attr ("type") and element['type'] != "hidden"): for attr in self.known_attribute_list: if element.has_attr(attr): locator = self.guess_xpath (guessable_element,attr,element) #"Xpath created" if len(driver.find_elements_by_xpath (locator))==1: result_flag = True print (locator.encode('utf-8')) break elif guessable_element == 'button' and element.getText(): button_text = element.getText() if element.getText() == button_text.strip(): locator = xpath_obj.guess_xpath_button (guessable_element,"text()",element.getText()) else: locator = xpath_obj.guess_xpath_using_contains (guessable_element,"text()",button_text.strip()) if len(driver.find_elements_by_xpath(locator))==1: result_flag = True print (locator.encode('utf-8')) break except Exception as e: print ("Exception when trying to generate xpath for:%s"%guessable_element) print ("Python says:%s"%str(e)) |
Check for uniqueness of the generated Xpath
XPath should have only one candidate element which means the XPath should output one and only one element. To make sure that the given XPath returns a single element we are checking the length of the elements for each locator in the generate_xpath() method and if it is greater than 1 checking for another attribute. In this way we are making sure that the XPath is unique.
In the above generate_xpath() code, we are calling three different methods guess_xpath(), guess_xpath_button() and guess_xpath_using_contains(). We shall discuss in detail about these methods in below sections.
guess_xpath()
The method guess_xpath(), accepts three arguments namely tag, attr, element, and XPath is guessed based on these arguments. To handle Unicode errors due to foreign Unicode characters we used encode() and join() functions.
The method guess_xpath(), accepts three arguments namely tag, attr, element, and XPath is guessed based on these arguments. To handle Unicode errors due to foreign Unicode characters we used encode() and join() functions.
guess_xpath_button()
We are using method guess_xpath_button() to check button.getText() condition. There will be situations, where you may not able to use any HTML property rather than text present in the element. text() function helps us to find the element based on the text present in the element. Since text() is a method, it does not need ‘@’ symbol as in case of an attribute.
We are using method guess_xpath_button() to check button.getText() condition. There will be situations, where you may not able to use any HTML property rather than text present in the element. text() function helps us to find the element based on the text present in the element. Since text() is a method, it does not need ‘@’ symbol as in case of an attribute.
guess_xpath_using_contains()
In some cases, we may have to use ‘contains‘ function which helps the user to find the element with partial values, or dynamically changing values, ‘contains‘ verifies matches with the portion of the value for text for which we don’t need the complete text but need only part of the text. In our case, while we are testing our code with few URL’s we encountered few button tags with text having leading and trailing spaces in the text. In such cases, the XPath guessed using above guess_xpath() may not work. Hence we wrote another method called guess_xpath_using_contains() which uses ‘contains‘ function and generates XPath as shown below.
In some cases, we may have to use ‘contains‘ function which helps the user to find the element with partial values, or dynamically changing values, ‘contains‘ verifies matches with the portion of the value for text for which we don’t need the complete text but need only part of the text. In our case, while we are testing our code with few URL’s we encountered few button tags with text having leading and trailing spaces in the text. In such cases, the XPath guessed using above guess_xpath() may not work. Hence we wrote another method called guess_xpath_using_contains() which uses ‘contains‘ function and generates XPath as shown below.
How we tested this utility
We tested this Utility with almost 50 different pages which have multiple input and button fields. We also tested with pages which don’t have any text or button fields. We came across few issues which we figured and fixed.
One of the issues we encountered is that when we ran the script in Windows, the input() command ran properly but when we ran it in Git Bash, we noticed there is a time gap in running input() command. The key to the problem is windows console prints text to the screen as soon as possible, while mingw (git bash) will wait until the application tells it to update the screen. To fix this, you can use -u flag interpreter command while running the script, which will stop python from buffering output in git bash as shown below.
below.
Screenshot of result of AutoLocatorsgenarator.py