Robots.txt

WebDynamic™ Web Glossary

Robots.txt is a file on a web sites root directory which spiders are supposed to read to determine which parts of a website they may or may not visit.

View the most recent additions to our portfolio

Our clients include some of Australia's most highly regarded organisations.

Enquire with one
of our consultants