Hello everyone. In this video I will be talking about a very interesting topic and concept, which is how web crawlers of search engines analyze a website content and index them in appropriate search results. We will see what does this “robots.txt” file actually does, its working, advantages and disadvantages, what does it contain, and how penetration testers can get help from this for their work in information gathering or vulnerability scanning as well. SO CLEAR THIS COMMON DOUBT AND GET STARTED!!
Follow this link to join my WhatsApp group: https://chat.whatsapp.com/3brcdlXRDiIBApiTACLqyo
Increase your GK with my Own Quiz App : https://play.google.com/store/apps/details?id=com.bittentech.quizz
Follow me on Instagram : https://instagram.com/ansh.98
Like me on Facebook : https://linksad.net/qUUBB
If you liked my video, please don’t forget to press the like button and subscribe to my youtube channel. I will be posting videos on cyber security, ethical hacking and technology. Also some interesting tricks
and techniques very soon so stay tuned and please SUBSCRIBE to my channel and thanks for watching….:-)