BLOG

SOMETIMES, WE EXPRESS!

How can a messed up website architect

  • 01/12/2017

The robots.txt file along with the sitemap is majorly responsible to create first impressions with the search engine web scraping and web crawling bots. An incorrect sitemap can make the web crawlers go crazy hunting for the pages they wish to index and may even end up indexing unfavourably.

Here’s a brief list of things that could be messed up when the bots come knocking at your sitemap-

  1. Format inconsistencies
    While there could be a horde of permutat.........
Read More

Why search engines are not web crawling your website/blog? Part 2

  • 30/11/2017

In the earlier post in this series, we had published the various popular types and levels of web crawlers followed by reasons why your blog or website is not being as popular as you expected it to be in the first place. Since we had readers really reach out to know more about why do their websites not come up as search results, this is a continuation to the earlier post in the series

In case you missed reading part 1, you can Read More


Tips & Tricks to Steer Your Web Crawling & Scraping Journey Clear of Legal Sludge

  • 26/11/2017

In the last few posts, we have establish the ground reality for various legal aspects of web scraping, data extraction & web crawling. While only the web administrator can decide how deep into the legal sludge does he wish to drag you, there are a few guidelines which can help you steer meaningfully clear of legal troubles.

These guidelines here are no guarantee for steering clear of legal troubles, it is always advisable to seek professional legal counsel to stay on the right s.........

Read More

Tips & Tricks to Steer Your Web Crawling & Scraping Journey Clear of Legal Slludge

  • 26/11/2017

In the last few posts, we have establish the ground reality for various legal aspects of web scraping, data extraction & web crawling. While only the web administrator can decide how deep into the legal sludge does he wish to drag you, there are a few guidelines which can help you steer meaningfully clear of legal troubles.

These guidelines here are no guarantee for steering clear of legal troubles, it is always advisable to seek professional legal counsel to stay on the right s.........

Read More


Get A Quote