Sunday, January 8, 2012

Finding the broken links in a webpage using Selenium


We heard about some Firefox plug-ins to find broken links in a webpage, like Link Checker, Xenu and etc. We need to install these plug-ins with Firefox browser and find the broken URLs or 404 pages.

We can write the Selenium script for the same functionality. How can we do that?
  • We need to find the number of links available on the page
  • We need to track each and every link
  • Finally we can get the response code for each and every URL or link with the help of HttpURLConnection Class and getResponseCode method.
How to find the number of links on the page?

We can find the number using selenium.getXpathCount("//a").intValue() method.

selenium=new DefaultSelenium("localhost", 4444, "*firefox", "http://www.yahoo.com");
selenium.start();
selenium.open("/");
int linkCount = selenium.getXpathCount("//a").intValue();

How to track each and every link on the page?

We can use the for loop and track the links one by one using this.browserbot.getUserWindow().document.links[] method. This will return the complete properties of the <a> tag of each URL. Then we can use selenium.getEval() method to extract only the HREF part of the <a> tag.

for (int i = 0; i < linkCount; i++) 
   {
     
       currentLink = "this.browserbot.getUserWindow().document.links[" + i + "]";
       temp = selenium.getEval(currentLink + ".href");

             }

How to find out the response code of the URL?

We can use HttpURLConnection Class and getResponseCode method for finding the reponse code of the URL.

public static int getResponseCode(String urlString) throws MalformedURLException, IOException {
   URL u = new URL(urlString); 
   HttpURLConnection huc =  (HttpURLConnection)  u.openConnection(); 
   huc.setRequestMethod("GET"); 
   huc.connect(); 
   return huc.getResponseCode();
}

Find out the complete Selenium Script below:

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.SeleneseTestBase;

public class BrokenURL extends SeleneseTestBase {
public int invalidLink;
String currentLink;
String temp;
public DefaultSelenium selenium;
@BeforeMethod
public void setUp() throws Exception
{
selenium=new DefaultSelenium("localhost", 4444, "*firefox", "http://www.yahoo.com");
selenium.start();
}
@Test
public void testUntitled() throws Exception {
FileOutputStream fout = new FileOutputStream ("broken_links.txt", true);
invalidLink=0;
selenium.open("/");
int linkCount = selenium.getXpathCount("//a").intValue();
   
new PrintStream(fout).println("URL : " + selenium.getLocation());
new PrintStream(fout).println("--------------------------------------------");
   for (int i = 0; i < linkCount; i++) 
   {
    int statusCode=0;
   
       currentLink = "this.browserbot.getUserWindow().document.links[" + i + "]";
       temp = selenium.getEval(currentLink + ".href");
       statusCode=getResponseCode(temp);
       if (statusCode==404)
       {
        new PrintStream(fout).println(selenium.getEval(currentLink + ".href") + " "+ statusCode);
        invalidLink++; 
       }
   }
   new PrintStream(fout).println("Total broken Links = " + invalidLink);
   new PrintStream(fout).println(" ");
fout.close();
   System.out.println(currentLink);
   System.out.println(temp);
}
public static int getResponseCode(String urlString) throws MalformedURLException, IOException {
   URL u = new URL(urlString); 
   HttpURLConnection huc =  (HttpURLConnection)  u.openConnection(); 
   huc.setRequestMethod("GET"); 
   huc.connect(); 
   return huc.getResponseCode();
}


public void tearDown()
{
selenium.close();
selenium.stop();
}
   
}

The above script will identify all the broken links(if any) in yahoo.com and store the 404 URLs in a notepad file called broken_links.txt. If you want to check the broken links for N number of URLs, you can pass the parameters through Data Provider concept or Excel sheet using JXL package.

85 comments:

  1. Hi veera,

    Can u pls provide the code for broken links in selenium 2

    Thanks,
    Hema

    ReplyDelete
  2. Is it at all possible to extendedd this to check 404s on images, JS and CSS (including any images loaded from within CSS files)?

    ReplyDelete
  3. That's nice, but what shall we do if the website requires logging in? We can log in with Selenium and get all the links, but how to modify getResponseCode method to handle such pages?

    ReplyDelete
  4. How can I use this on a HTTPS application? i get over and over all the workarounds to avoid certificates, but i got the same error message:

    javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching xxxx.xxxxxxxxxx.xxxx found

    ReplyDelete
  5. Hi Veera,

    I tried with the above code but did not work for me. Can Is it mandatory to use "DefaultSelenium"?

    Thanks,
    Sudhansu

    ReplyDelete
  6. It's also possible to add some method that will check only specific links, e.g. with German domain .de, or will skip mail links.
    more ditails here http://www.skillim.com/check-broken-links-on-a-page-using-selenium/

    ReplyDelete
  7. Thanks for Sharing this valuble information and itis useful for me and Selenium learners.We also provides the best Online Selenium Training classes

    ReplyDelete
  8. The information which you provided seems to be useful. Thanks.
    Excel Training

    ReplyDelete
  9. Nice information. Our institute Is the Leading online institute in Hyderabad, India to provide SAS online trainings with the real time experts and certified professionals. alot SAS Online Training.

    ReplyDelete
  10. This helps to locate broken links on website to find the flaws and rectifying it by just simply entering the URL.It is vary useful check on
    broken link finder

    ReplyDelete
  11. Thanks a lot we've it's share for the grateful article.

    Informatica Online Training

    ReplyDelete
  12. Selenium WebDriver may not be the best tool for the job.Because everything happens through the browser which makes things very slow.I was looking a while ago into ways of making Selenium WebDriver scripts faster.One of the ideas was to use HTTP Parsing Libraries for automating scripts that should be fast.

    Selenium Training in Chennai

    ReplyDelete
  13. very helpful..:)

    do check out
    http://seleniumtestingcom.blogspot.in/

    for more info on selenium

    ReplyDelete
  14. You are doing a great job. You inspire me to write for other. Thank you very much. I would like to appreciate your work for good accuracy and got informative knowledge from here. Selenium Training in Chennai

    ReplyDelete
  15. Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information... Java Training in Chennai | Blue prism Training in Chennai

    ReplyDelete
  16. you are able to take the help of experts making a call QuickBooks Payroll Support Phone Number Well! If you’re not in a position to customize employee payroll in.

    ReplyDelete
  17. Do you think you're confident about it? Or even, this could be basically the right time to get the QuickBook Customer Support Number. We now have trained staff to soft your issue. Sometimes errors may possibly also happen as a result of some small mistakes. Those are decimals, comma, backspace, etc.

    ReplyDelete
  18. This might protect the contents. It truly is great at fringe benefits. This program is SARS compliant. It is in reality good in tax statements. Submission of tax is not hard with QuickBooks Payroll Tech Support Number.

    ReplyDelete
  19. The good thing of the QuickBooks Customer Support Number is the fact that not merely you’ll prepare you to ultimately resolve your problems nevertheless you may be often recognized by our technicians and he/she could well keep updating you concerning your problems. you may have an entire information what the problem your package is facing.

    ReplyDelete
  20. Our experts deliver technology-infused support and solutions for the best customer experience. Our team consists of certified and qualified experts who have years of experience and in-depth knowledge of QuickBooks Enterprise Technical Support and associated software.

    ReplyDelete
  21. QuickBooks encounter a sum of undesirable and annoying errors which keep persisting over time if you don't resolved instantly. Considered one of such QuickBooks Tech Support Number issue is Printer issue which mainly arises due to a number of hardware and software problems in QuickBooks, printer or drivers.

    ReplyDelete
  22. Sometimes it’s normal for users to have some performance-related issues in your QuickBooks due to your virus in your personal computer or some other computer related issues.In that case, you are annoying and you need some support to help you out. Here you can easily call us at our QuickBooks Payroll Support Phone Number to get instant help from our technical experts. However, if you would like speak to us quickly, you can easily reach us at QuickBooks Support Phone Number and all your queries may be solved instantly.

    ReplyDelete
  23. QuickBooks Payroll has emerged one of the better accounting software that has had changed the meaning of payroll. Quickbooks Payroll Support Number will be the team that provide you Quickbooks Payroll Support. This software of QuickBooks is sold with various versions and sub versions.

    ReplyDelete
  24. QuickBooks Support Number advisors are certified Pro-advisors’ and has forte in furnishing any type of technical issues for QuickBooks. These are generally expert and certified technicians of the domains like QuickBooks accounting,QuickBooks Payroll, Point of Sales, QuickBooks Merchant Services and Inventory issues to provide 24/7 service to our esteemed customers. QuickBooks payroll Services provide approaches to all your valuable QuickBooks problem and in addition assists in identifying the errors with QuickBooks data files and diagnose them thoroughly before resolving these issues.

    ReplyDelete
  25. QuickBooks Tech Support Number is an application which includes made payroll a simple snap-of-fingers task. You'll be able to quite easily and automatically calculate the tax for your employees. It is an absolute software that fits your organization completely.

    ReplyDelete
  26. Keeping these records in your records will likely be quite advantageous to maintain your QuickBooks Payroll Support Phone Number account along with information with respect to federal along with state agencies.

    ReplyDelete
  27. The principal intent behind QuickBooks Support Phone Number is to offer the technical help 24*7 so as in order to prevent wasting your productivity hours. This will be completely a toll-free QuickBooks client Service variety that you won’t pay any call charges. Needless to say, QuickBooks is one among the list of awesome package into the company world. The accounting area of the many companies varies based on this package. You will discover so many fields it covers like creating invoices, managing taxes, managing payroll etc. However exceptions are typical over, sometimes it makes the down sides and user wants QuickBooks Support. Our professionals are terribly dedicated and might solve your entire issues without the fuss. In the event that you call, you will be greeted by our client service representative when taking all of your concern he/she will transfer your preference in to the involved department. The good thing would be the fact that not just you’ll prepare yourself to resolve your problems nevertheless you may be often acknowledged by our technicians and he/she could well keep updating you concerning your problems.

    ReplyDelete
  28. QuickBooks Error 3371 could also be a result of multiple problems e.g. something associated with other the different parts of your QuickBooks software or Operating system as mentioned underneath

    ReplyDelete
  29. Intuit QuickBooks Payroll Support Phone Number accounting software enables you to prepare your invoices, manage your business payrolls, track your business inventory, control cash flow, and be tax-ready. Intuit Online Payroll is the better option for companies looking forward to automating their accounting solutions and take their company to new heights.

    ReplyDelete
  30. QuickBooks Enterprise Support contact number is assisted by an organization this is certainly totally dependable. It is a favorite proven fact that QuickBooks Enterprise Support Phone Number has had about plenty of improvement in the area of accounting. In the long run quantity of users and selection of companies that can be chosen by some one or the other

    ReplyDelete
  31. QuickBooks Enterprise Support Phone Number is accounting software, which will be a cloud-based application produced by Inuit Inc. As a matter of fact, the program has been developed with all the intention of keeping a secure record of financial needs for the business.

    ReplyDelete
  32. Your Technical support team is merely an individual tap away, dial our QuickBooks Tech Support Number and experience our best hassle-free tech support team.

    ReplyDelete
  33. So now you have become well tuned directly into advantages of QuickBooks online payroll in your company accounting but because this premium software contains advanced functions that may help you along with your accounting task to complete, so you may face some technical errors while using the QuickBooks payroll solution. In that case, Quickbooks Support Number provides 24/7 make it possible to our customer.

    ReplyDelete
  34. We suggest you to definitely join our services just giving ring at toll-free QuickBooks Enterprise Support make it possible for one to fix registration, installation, import expert and plenty of other related issues into the enterprise version.

    ReplyDelete
  35. Let’s see some awesome features which could have caused it is so popular. If you should be also a QuickBooks user and desires to find out more concerning this software you may take a look at the QuickBooks Enterprise Support.

    ReplyDelete
  36. But don’t worry we are always here to aid you. As you are able to dial our QuickBooks Payroll Support Number. Our QB online payroll support team provide proper guidance to solve all issue associated with it. I'll be glad to help you.

    ReplyDelete
  37. An entire support against all kinds of QuickBooks Pro Support Number related issues is provided by our experts utilizing the demonstration of easy solutions. Users get help for: Troubleshooting QuickBooks error codes and messages Updating, creating, and rebuilding data files on QuickBooks.

    ReplyDelete
  38. In case the company also uses QuickBooks Phone Number Tech Support software to handle your sales, inventory and customers’ information, we have experts who can assist you to maintain this system too. Aside from troubleshooting, we could help you put up custom reporting,

    ReplyDelete
  39. At site name, the QQuickBooks Payroll Support Number team assists you to in resolving every error that hinders the performance and speed with this work. Many of us studies every issue beforehand and offers you the optimised solution. If you come with any issue which many of us is just not conscious of then it`s not most likely a challenge for the team as it's quick and sharp to locate out from the issue and resolving it straight away.

    ReplyDelete
  40. QuickBooks Enterprise Technical Support Number is an accounting solution that is favorable for small to mid-sized businesses encapsulating all of the sections like construction, distribution, manufacturing, and retail. It gives a multi-users feature that means it is feasible for many users to use exactly the same computer to enable faster workflow.

    ReplyDelete
  41. Very often client faces some typically common issues like he/she isn’t willing to open QuickBooks package, it is playing terribly slow, struggling to install and re-install, a challenge in printing checks or client reports. We intend to supply you with the immediate support by our well- masterly technicians. A group of QuickBooks Payroll Tech Support Number dedicated professionals is invariably accessible for you personally so as to arranged every one of your problems in an effort that you’ll be able to do your work while not hampering the productivity.

    ReplyDelete
  42. Get outstanding possibilities for QuickBooks towards you right away! Without having any doubts, QuickBooks has revolutionized the process of doing accounting that's the core strength for small as well as large-sized businesses. QuickBook Support is assisted by our customer support specialists who reply to your call instantly and resolve all of your issues on the spot.

    ReplyDelete
  43. Thus, there's no possibility for data getting violated. You should QuickBooks Technical Support when it comes to a number of software issues. The satisfaction can be high class with us. It is possible to call us in several ways. You can journey to our website today. It is time to get the best help.

    ReplyDelete
  44. QuickBooks Payroll has emerged the best accounting software which includes had changed this is of payroll. QuickBooks Payroll Tech Support Phone Number would be the team that offer you Quickbooks Payroll Support. This software of QuickBooks comes with various versions and sub versions. Online Payroll and Payroll for Desktop is the two major versions and they're further bifurcated into sub versions. Enhanced Payroll and Full-service payroll are encompassed in Online Payroll whereas Basic, Enhanced and Assisted Payroll come under Payroll for Desktop.

    ReplyDelete

  45. How to contact QuickBooks Payroll support?
    Different styles of queries or QuickBooks related issue, then you're way in the right direction. You simply give single ring at our toll-free intuit QuickBooks Payroll Support number . we are going to help you right solution according to your issue. We work on the internet and can get rid of the technical problems via remote access not only is it soon seeing that problem occurs we shall fix the same.

    ReplyDelete
  46. However, if you prefer are looking for HP Printer troubleshooting, don’t forget to dial the HP Printer Tech Support. The telephone number immediately connects you to definitely the pros who are available for you round the clock to give you an instant help to correct HP.

    ReplyDelete
  47. For most for the small or mid-sized businesses organizations, it is always been the QuickBooks Support most challenging task to easily manage the business accounts in a suitable and genuine way by simply locating the appropriate solutions.

    ReplyDelete

  48. Any user can try to find available these days payroll update when you head to “employee” menu, selecting “get QuickBooks Payroll Support Phone Number updates” after which option “update”.

    ReplyDelete
  49. QuickBooks Customer Support Phone Number have a team of professionals that have extensive QB expertise and knowledge on how to tailor this software to any industry. Having performed many QB data conversions as well as other QB engagements, we have the experience as you are able to rely on.

    ReplyDelete
  50. Intuit QuickBooks Support is accounting software, which is a cloud-based application produced by Inuit Inc. In fact, the application has been developed with the intention of keeping a safe record of financial needs for the business. Additionally, it is a user-friendly accounting software; an easy task to maintain; assisting the company in keeping the records of financial transactions, and a whole lot more features.

    ReplyDelete
  51. Very often client faces some typically common issues like he/she isn’t willing to open QuickBooks package, it is playing terribly slow, struggling to install and re-install, a challenge in printing checks or client reports. We intend to supply you with the immediate support by our well- masterly technicians. A group of QuickBooks Support Phone Number dedicated professionals is invariably accessible for you personally so as to arranged every one of your problems in an effort that you’ll be able to do your work while not hampering the productivity.

    ReplyDelete
  52. problem in upgrading the program into the newer version so that you could avail the most recent Intuit QuickBooks Support features, trouble in generating advanced reports, difficulty with opening company file in multi-user mode and thus on and so forth.

    ReplyDelete
  53. If you are aa QuickBooks enterprise user, you'll be able to reach us out immediately at our QuickBooks Support contact number . QuickBooks technical help is present at our QuickBooks Support Phone Number dial this and gets your solution from our technical experts.

    ReplyDelete
  54. Quicken Support Phone Number

    Quicken Support Phone Number: +1-888-422-3444.
    Call Time: 24 Hours 7 Days, also available on holidays. Quicken is an accounting tool for bookkeeping purposes that assist clients in keeping track of their finances. It is a very capable and reliable finance tool and it is equipped with amazing money management functionality and …

    ReplyDelete
  55. lexmark Printer Support Phone Number

    Lexmark Printer Support Phone Number + 1-888-600-5222
    name in case you need help together with your Lexmark Printer. guide available for printing problems, scanning issues, connectivity troubles, and many more. based in 1991 Lexmark is most of the exceptional laser printer and imaging product makers within the world.

    ReplyDelete
  56. QuickBooks Enterprise Support Phone Number assists you to definitely overcome all bugs through the enterprise types of the applying form. Enterprise support team members remain available 24×7 your can buy facility of best services.

    ReplyDelete
  57. This where in fact the QuickBooks Support Phone Number comes into play. Once you obtain in touch with all the QuickBooks Support Phone Number team, our expert technicians provides you with the technical assistance that you want to get rid of whatever problems that could be bogging your system down.

    ReplyDelete
  58. This software is helpful to track the unbilled time and expenses of your client and employee. User can set distinct billing rates and can pay the employees very easily. QuickBooks Enterprise helps you to establish your business from scratch, right from the cost to advertise your business proposal. If you are looking for assistance, dial on QuickBooks Enterprise Tech Support Number and get all issues fixed by the highly-experienced team. Our team of QuickBooks Enterprise Support is available throughout the day for helping customers.

    ReplyDelete
  59. Any errors connected with QuickBooks software may damage your computer data you could fix them through QuickBooks Premier Support Phone Number and QuickBooks 24/7 Support will always be there to greatly help. We solve, manage, which help you to definitely cope with any technical problems and guarantees that this program is effective.

    ReplyDelete
  60. You can reach the support experts of QuickBooks Payroll Support Number, without the necessity to go out your working environment. The experts do not visit your working environment either.

    ReplyDelete
  61. QuickBooks support phone number can be contacted to learn the ways to create an automatic backup to save all your employee-related data from getting bugged or lost at any circumstances. The QuickBooks support phone number is toll-free and the professional technicians handling your support call can come up with an immediate solution that can permanently solve the glitches.Connecting with the executives through the QuickBooks Support Phone Number takes less than thirty seconds of the time period.

    ReplyDelete
  62. The world-wide-web is stuffed with faux numbers WHO decision themselves the QuickBooks Support Phone Number Provider. you’ll value more highly to dial their variety however that would be terribly risky. you’ll lose your QuickBooks Company file or the code itself.

    ReplyDelete
  63. From setting up QuickBooks Payroll Support to moving your organization information and settling issues and mistake messages, we help you precisely make use of the business accounting software and make use of it.

    ReplyDelete
  64. Through these business owners or entrepreneurs can certainly manage their business (financial and accounting information) accurately. But sometime when using it you may need appropriate and effective Quickbooks help by either you or your QuickBooks online accountant to get the best and effective usage of this software. The QuickBooks Tech Support Phone Number is toll-free additionally the professional technicians handling your support call can come up with an immediate solution that may permanently solve the glitches.

    ReplyDelete
  65. If you wish to mitigate all the aforementioned errors by yourself, you can acquire QuickBooks File Doctor. This application will conduct a scan and repair most technical errors automatically. Get quick help by contacting the QuickBooks Support.Get quick help by contacting the QuickBooks customer care contact number.

    ReplyDelete
  66. All this is done without compromising with all the quality of services because nothing seems good in the event that work is not done. Our QuickBooks Tech Support team is enthusiastic and makes best usage of its experience. They just do not let go any issue even if it is fairly complex.

    ReplyDelete
  67. We have a team this is certainly extremely supportive and customer friendly.Our customer service executives at QuickBooks Support Phone Number try not to hesitate from putting extra efforts to provide you with relief from the troubles due to QB Payroll errors.We take good care of our customers and bend towards backward to please them with our exuberant performance. All this is done without compromising with all the quality of services because nothing seems good in the event that work is not done.Our customer support team is enthusiastic and makes best usage of its experience. They just do not let go any issue even if it is fairly complex.

    ReplyDelete
  68. The QuickBooks Support Phone Number could be reached all through night and day as well as the technicians are particularly skilled to manage the glitches which are bugging your accounting process.

    ReplyDelete
  69. The QuickBooks Support Number is also used to eradicate your questions related to using various options that come with this remarkable product from QuickBooks.

    ReplyDelete
  70. Hi! Lovely post. I simply couldn’t leave your website before suggesting that I liked your work. I’m gonna be back continuously to check up on the new post. QuickBooks POS is one of the best versions of QuickBooks. You can avail instant help and support at QuickBooks POS Support Phone Number 1-855-236-7529. So, in case you come across any error then simply dial our QuickBooks Technical Support Phone Number 1-855-236-7529.
    Read more: http://bit.ly/2kgOogd

    ReplyDelete
  71. Awesome blog! Is your theme custom made or did you download it from somewhere? A design like yours with a few simple tweeks would really make my blog shine. Please let me know where you got your theme. Kudos Miss Samsung Fascinate company website

    ReplyDelete

  72. QuickBooks is the greatest choices as an accounting and financial software for small and medium-size business to control their company data in a simple and organized way. QuickBooks software contains several pre-created reports that may be customized to produce information from the company data file. If you want to Resolve QuickBooks Error 9999 then you may contact our ProAdvisors.

    ReplyDelete
  73. Awesome blog! Is your theme custom made or did you download it from somewhere? A design like yours with a few simple tweeks would really make my blog shine.

    AWS training in Chennai

    AWS Online Training in Chennai

    AWS training in Bangalore

    AWS training in Hyderabad

    AWS training in Coimbatore

    AWS training


    ReplyDelete
  74. Very Informative blog thank you for sharing. Keep sharing.

    Best software training institute in Chennai. Make your career development the best by learning software courses.

    microsoft azure training institutes in chennai
    devops training in chennai
    power bi training in chennai

    ReplyDelete
  75. Thanks for one marvelous posting! I enjoyed reading it; you are a great
    author. I will make sure to bookmark your blog and may come back
    someday. I want to encourage that you continue your great posts, have
    a nice weekend!
    oracle course in chennai
    ASP Dot Net Training in Chennai
    C Sharp Training in Chennai

    ReplyDelete