{"id":238308,"date":"2025-07-09T10:06:26","date_gmt":"2025-07-09T10:06:26","guid":{"rendered":"https:\/\/wordpress.org\/plugins\/llms-txt-ai-website-content-crawling\/"},"modified":"2025-07-09T10:05:53","modified_gmt":"2025-07-09T10:05:53","slug":"llms-txt-ai-website-content-crawling","status":"publish","type":"plugin","link":"https:\/\/tg.wordpress.org\/plugins\/llms-txt-ai-website-content-crawling\/","author":11913801,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_crdt_document":"","version":"1.0","stable_tag":"1.0","tested":"6.8.5","requires":"","requires_php":"7.2","requires_plugins":null,"header_name":"LLMs.txt - AI Website Content Crawling","header_author":"Magazine3","header_description":"Generates a structured llms.txt file based on selected post types and settings.","assets_banners_color":"","last_updated":"2025-07-09 10:05:53","external_support_url":"","external_repository_url":"","donate_link":"","header_plugin_uri":"","header_author_uri":"","rating":0,"author_block_rating":0,"active_installs":0,"downloads":355,"num_ratings":0,"support_threads":0,"support_threads_resolved":0,"author_block_count":0,"sections":["description","installation","faq","changelog"],"tags":{"1.0":{"tag":"1.0","author":"magazine3","date":"2025-07-09 10:05:53"}},"upgrade_notice":[],"ratings":[],"assets_icons":[],"assets_banners":[],"assets_blueprints":{},"all_blocks":[],"tagged_versions":["1.0"],"block_files":[],"assets_screenshots":[],"screenshots":{"1":"Admin settings screen to select post types.","2":"Generated llms.txt output example."},"jetpack_post_was_ever_published":false},"plugin_section":[],"plugin_tags":[244715,182970,244604,12753,186],"plugin_category":[],"plugin_contributors":[169462],"plugin_business_model":[],"class_list":["post-238308","plugin","type-plugin","status-publish","hentry","plugin_tags-ai-crawling","plugin_tags-content-control","plugin_tags-llms-txt","plugin_tags-robots-txt","plugin_tags-seo","plugin_contributors-magazine3","plugin_committers-magazine3"],"banners":[],"icons":{"svg":false,"icon":"https:\/\/s.w.org\/plugins\/geopattern-icon\/llms-txt-ai-website-content-crawling.svg","icon_2x":false,"generated":true},"screenshots":[],"raw_content":"\n

LLMs.txt - AI Website Content Crawling<\/strong> helps you generate a custom llms.txt<\/code> file to control how AI language models (LLMs) and search engines crawl your site content.<\/p>\n\n

Key Features<\/h3>\n\n
    \n
  • Automatically generate a valid llms.txt<\/code> file.<\/li>\n
  • Choose which post types are included for AI access.<\/li>\n
  • Compatible with SEO and privacy-conscious setups.<\/li>\n
  • Easy to use with minimal configuration.<\/li>\n<\/ul>\n\n

    This plugin is ideal for publishers, bloggers, and businesses looking to control their content visibility to AI systems and bots.<\/p>\n\n\n

      \n
    1. Upload the plugin folder to the \/wp-content\/plugins\/<\/code> directory.<\/li>\n
    2. Activate the plugin through the 'Plugins' menu in WordPress.<\/li>\n
    3. Go to Settings > LLMs.txt<\/strong> to configure which post types to include.<\/li>\n
    4. Visit https:\/\/yoursite.com\/llms.txt<\/code> to see the output.<\/li>\n<\/ol>\n\n\n
      \n

      What is `llms.txt`?<\/h3><\/dt>\n

      It is a proposed standard for signaling content access preferences to large language models and AI crawlers, similar to robots.txt<\/code>.<\/p><\/dd>\n

      Does this replace `robots.txt`?<\/h3><\/dt>\n

      No. llms.txt<\/code> complements robots.txt<\/code> and focuses specifically on AI crawling policies.<\/p><\/dd>\n

      Can I control which content types are included?<\/h3><\/dt>\n

      Yes, the plugin allows you to select which post types (e.g., posts, pages, custom types) should be listed.<\/p><\/dd>\n\n<\/dl>\n\n\n

      1.0<\/h4>\n\n
        \n
      • Initial release.<\/li>\n<\/ul>","raw_excerpt":"Generate a structured llms.txt file to manage AI and search engine crawling access across your WordPress site.","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin\/238308","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin"}],"about":[{"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/types\/plugin"}],"replies":[{"embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/comments?post=238308"}],"author":[{"embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wporg\/v1\/users\/magazine3"}],"wp:attachment":[{"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/media?parent=238308"}],"wp:term":[{"taxonomy":"plugin_section","embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_section?post=238308"},{"taxonomy":"plugin_tags","embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_tags?post=238308"},{"taxonomy":"plugin_category","embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_category?post=238308"},{"taxonomy":"plugin_contributors","embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_contributors?post=238308"},{"taxonomy":"plugin_business_model","embeddable":true,"href":"https:\/\/tg.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_business_model?post=238308"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}