The Guys in Tech

Hello !  In my previous  article , we discussed about Nokigiri , a very useful HTML and XML parser i.e it helps in parsing or scraping the information available on a web page to use it in the Ruby program. But sometimes we just want to trigger some action , click button or automate the process on a website from a Ruby program.For example scheduling a social media post , automating a feed back (Yes , we are forced to give the feed back to the faculties on a web portal) or just submitting a form and many other features that are not provide by that particular website but you have to do it manually.This is where Mechanize comes to the rescue . It is one of the coolest ruby library that is very useful in scrapping as well as triggering any action on a web page. What's Mechanize ? According to the The Bastards Book of Ruby , "Sometimes, the challenge is navigating to the right page. This is the case with many websites that require you to fill out a form. It's not ...

Websites are made of web pages. Every webpage is made of html tags. And every webpage has a unique address called URL which stands for Uniform Resource Locator. Using which we can request the webpage html code from the server. Once we send the request, the server will give us the html file for the webpage and our browser translates the code into readable English. This is the process that takes place every time we browse the internet. In today’s post, we will see how to do that without the browser and rather through python. And also we will see how to grab all the links present on a website. A link is another URL. There can be many uses of doing this. For example, a web crawler does this all the time. It goes through the html code of a website and copies the links. And it visits those individual links and looks for other links in that page too. This is how the Google indexes pages. It crawls the whole of the internet every day. Python is very easy, as is this process. So let’s get star...