# See https://www.robotstxt.org/robotstxt.html User-agent: * # Search/filter URLs are an unbounded, near-duplicate space; the profile and # record pages are what should get the crawl budget. Disallow: /search? Disallow: /*?q= Disallow: /*?page= Disallow: /ads/ Sitemap: https://licensedirect.com/sitemaps/sitemap.xml.gz