Uses of Class
org.attribyte.api.http.util.RobotsTxt

Packages that use RobotsTxt

Package	Description
org.attribyte.api.http.util

Uses of RobotsTxt in org.attribyte.api.http.util

Fields in org.attribyte.api.http.util declared as RobotsTxt

Modifier and Type	Field	Description
static final RobotsTxt	RobotsTxt.NO_ROBOTS

Methods in org.attribyte.api.http.util that return RobotsTxt

Modifier and Type	Method	Description
static RobotsTxt	RobotsTxt.parse(String host, Client httpClient, String userAgent, Set<String> preserveAgents, org.attribyte.api.Logger logger)	Creates a robots.txt from the standard location (/robots.txt).