---
title: "AI クローラー役割分類および可視性エビデンス基準データセット"
description: "CC BY 4.0 ライセンスで公開される構造化データセット：主要 14 種の AI クローラー役割分類、4 層の帰属エビデンス基準、公開静的 GEO スコア配点を収録。"
canonical_url: "https://citationgraph.ai/ja/data/ai-crawler-and-visibility-taxonomy"
markdown_url: "https://citationgraph.ai/ja/data/ai-crawler-and-visibility-taxonomy.md"
locale: "ja"
entity_id: "https://citationgraph.ai/#software-application"
parent_organization: "https://www.global-gravity.com/#organization"
sibling_product: "https://getgravityads.ai/#software-application"
---

# AI クローラー役割分類および可視性エビデンス基準データセット

> CitationGraph は AI ボット通信を「学習用クローラー」「AI 検索インデックス」「ユーザープロンプト起点のリアルタイム取得」の 3 役割に分類し、4 層のファーストパーティ証跡モデルと 4 次元均等配点（インフラ 25%・メタデータ 25%・構造化データ 25%・AI ガイダンス 25%）で技術的準備度を定義します。

- **License**: https://creativecommons.org/licenses/by/4.0/
- **CSV Download**: https://citationgraph.ai/data/ai-crawler-and-visibility-taxonomy.csv
- **JSON Download**: https://citationgraph.ai/data/ai-crawler-and-visibility-taxonomy.json

## 1. 主要 AI クローラー＆検索エージェント役割分類（14 種）

| Bot Token | Operator | Role | robots.txt Token | Signal Layer | Verification |
| :--- | :--- | :--- | :--- | :--- | :--- |
| `GPTBot` | OpenAI | `model_training` | `GPTBot` | `server_crawler_discovery` | Published IP CIDR ranges + User-Agent |
| `OAI-SearchBot` | OpenAI | `search_index` | `OAI-SearchBot` | `server_crawler_discovery` | Published IP CIDR ranges + User-Agent |
| `ChatGPT-User` | OpenAI | `live_user_retrieval` | `ChatGPT-User` | `live_retrieval_fetch` | Published IP CIDR ranges + User-Agent |
| `ClaudeBot` | Anthropic | `model_training` | `ClaudeBot` | `server_crawler_discovery` | User-Agent + ASN attribution |
| `Claude-SearchBot` | Anthropic | `search_index` | `Claude-SearchBot` | `server_crawler_discovery` | User-Agent + ASN attribution |
| `Claude-User` | Anthropic | `live_user_retrieval` | `Claude-User` | `live_retrieval_fetch` | User-Agent + request header pattern |
| `PerplexityBot` | Perplexity AI | `search_index` | `PerplexityBot` | `server_crawler_discovery` | Published IP JSON list + User-Agent |
| `Perplexity-User` | Perplexity AI | `live_user_retrieval` | `Perplexity-User` | `live_retrieval_fetch` | Published IP JSON list + User-Agent |
| `Google-Extended` | Google | `model_training` | `Google-Extended` | `server_crawler_discovery` | Reverse/Forward DNS googlebot.com + policy token |
| `GoogleOther` | Google | `search_index` | `GoogleOther` | `server_crawler_discovery` | Reverse/Forward DNS googlebot.com |
| `Applebot-Extended` | Apple | `model_training` | `Applebot-Extended` | `server_crawler_discovery` | Reverse/Forward DNS applebot.apple.com + policy token |
| `Amazonbot` | Amazon | `search_index` | `Amazonbot` | `server_crawler_discovery` | Reverse/Forward DNS crawl.amazonbot.amazon |
| `Meta-ExternalAgent` | Meta | `model_training` | `Meta-ExternalAgent` | `server_crawler_discovery` | User-Agent + Meta ASN verification |
| `CCBot` | Common Crawl | `model_training` | `CCBot` | `server_crawler_discovery` | User-Agent + Common Crawl AWS egress metadata |

## 2. 4層ファーストパーティ AI 可視性エビデンスモデル

| Layer ID | Layer Name | Signal Source | Deterministic Scope |
| :--- | :--- | :--- | :--- |
| `L1_server_crawler_truth` | Server Crawler & Retrieval Telemetry | Edge/runtime HTTP request headers, status codes, and verified bot identity | Proves crawler discovery and live answer-engine page fetches without client JS |
| `L2_citation_surface_readiness` | Citation Surface & Entity Graph Readiness | Canonical HTML, JSON-LD @id graph, llms.txt, and .md dual-stack mirrors | Evaluates structural eligibility for verifiable citations; does not guarantee citation |
| `L3_verified_ai_referral_session` | First-Party AI Referral Session Attribution | Document referrer, UTM/citation parameters, and session landing page | Separates human clicks arriving from AI answer engines from generic organic/direct visits |
| `L4_connected_outcome_context` | Connected Search & Conversion Context | Authorized GSC query matrices, waitlist/lead events, and commerce outcomes | Available in connected workspaces only; never inferred from public static previews |

## 3. 公開静的 GEO プレビュースコア配点（100点満点）

| Dimension | Weight | Evaluated Signals | Boundary Note |
| :--- | :--- | :--- | :--- |
| `infrastructure` | 25% | HTTPS enforcement, HTTP 200 status, robots.txt crawl rules, XML sitemap presence | Public static scan dimension; excludes private server logs |
| `metadata` | 25% | Title & meta description boundaries, self-referencing canonical, hreflang reciprocity, Open Graph | Public static scan dimension; checks single-URL HTML head tags |
| `structured_data` | 25% | Valid JSON-LD syntax, Organization/SoftwareApplication/Dataset @id entities, FAQ/Article schema | Public static scan dimension; validates machine-parsable entity definitions |
| `ai_guidance` | 25% | llms.txt / llms-full.txt availability, Markdown alternate Link headers, clean section headings | Public static scan dimension; does not represent hosted multi-engine citation tracking |
