The penetration tester Thiebauld Weksteen has published an interesting analysis to explaine the importance of robots.txt for the hacking activities. Thiebauld Weksteen, a penetration tester from Melbourne is advising system administrators that robots.txt can give precious details to hackers, when it comes to attacks, because robots.txt as the capability to tell search engines which directories […]