Skip to main content
Datasource Metadata Schema defines the structure of metadata that every resource in your datasource must follow. This ensures consistency and enables powerful filtering capabilities.

Metadata Schema

What is a Metadata Schema?

A Metadata Schema defines a JSON structure that every resource in your datasource must follow. It’s like a contract:
  • Datasource with metadata schema:
    • Field 1: department (type: str, required)
    • Field 2: priority (type: int, optional)
    • Field 3: version (type: str, optional)
  • Every resource must provide:
    • department (string)
    • priority (number) — optional, can be empty
    • version (string) — optional, can be empty

Schema Field Definition

Each field in a metadata schema has:

Example Metadata Schema

Creating a Metadata Schema

1

Go to Datasource

Navigate to the datasource you want to configure
2

Open Schema Configuration

Find Metadata Schema section in datasource settings
3

Define Fields

Add fields with name, type, and whether they’re optional
4

Set Required Fields

Mark critical fields as required (optional=false)
5

Save Schema

Schema is now enforced for all resources

Datasource Basic Properties

Name

Display name of the datasource

Description

Purpose and contents of the datasource

Type

Manual or Google Drive

Owner

Primary maintainer

Datasource Scope & Sharing

Primary Project

Project where datasource is created

Shared With Projects

Other projects that can access this datasource

Resource Count

Number of files, URLs, and content entries

Storage Size

Total size of all resources

Creating Datasource Metadata

Setting Up a New Datasource

1

Navigate to Datasources

Go to Datasources in your project
2

Create New Datasource

Click Create Datasource
3

Choose Type

Select Manual or Google Drive
4

Enter Basic Info

Fill in name and description
5

Configure Metadata

Set tags, visibility, and owner
6

Save

Datasource is created and ready

Metadata Fields Explained

Name

The display name for your datasource. Requirements:
  • Unique within your project
  • Clear and descriptive
  • 3-100 characters
  • No special characters
Examples:
Naming Conventions:

Description

A summary of what the datasource contains. Purpose:
  • Help team members understand content
  • Guide agent configuration
  • Document coverage
  • Explain use cases
Example:
What to Include:
  • Main topics covered
  • Update frequency
  • Intended users/agents
  • Maintenance status
  • Key features

Type

The datasource category:

Status

Track the datasource state:
Currently in use
  • Updated regularly
  • Used by agents
  • Primary reference

Tags

Categorize your datasources. Tag Examples:
Tag Management:
  • Use consistent naming (lowercase, kebab-case)
  • Limit to 5-10 tags per datasource
  • Reuse existing tags for consistency
  • Document tag meanings

Visibility

Control who can see and use the datasource:
Only you can see and manage
  • Not visible to others
  • Full control
  • Default for new datasources

Owner

The primary maintainer of the datasource. Responsibilities:
  • Keep content current
  • Add/remove resources
  • Update metadata
  • Monitor quality
  • Plan archival/deprecation
Changing Owner:

Resource Count

Shows how much content is in the datasource: Example: Customer Support Datasource

Storage Size

Total size of all resources in the datasource. Considerations:
  • Impacts retrieval speed
  • Affects storage quota
  • Consider splitting if very large
  • Archive old resources
Optimization (Large Datasource, 500 MB):
  • Archive old versions
  • Remove duplicates
  • Split by topic
  • Target: under 200 MB

Last Updated

When content was last modified. Information:
  • Date of last change
  • Who made the change
  • What was updated
  • Helps identify fresh content
Using Date:
  • Verify content is current
  • Plan updates
  • Track maintenance
  • Identify stale datasources

Agent Count

Number of agents using this datasource. Impact:
  • More agents = more usage
  • Higher importance
  • Wider impact of changes
  • More careful updates needed
Example: Customer Support FAQ (used by 8 agents)
  • Support Bot
  • Email Agent
  • Chat Assistant
  • Knowledge Base Bot
  • Escalation Agent
  • FAQ Agent
  • Billing Support
  • Technical Support

Editing Datasource Metadata

Update Information

1

Find Datasource

Navigate to the datasource list
2

Open Settings

Click Edit or settings icon
3

Modify Fields

Update name, description, tags, status, etc.
4

Save Changes

Click Save to apply updates
5

Confirm

Changes are immediately available

What Can Be Changed

Editable:
  • Name (with limitations)
  • Description
  • Tags
  • Status
  • Owner
  • Visibility
Not Editable:
  • Type (Manual vs Google Drive)
  • Created date (system-generated)
  • Resource count (updated automatically)

Using Metadata for Organization

Organizational Strategies

By Department:
By Content Type:
By Audience:
By Lifecycle:

Naming Convention

Establish consistent naming:

Datasource Organization Examples

Example 1: Support Team Structure

Example 2: Product Documentation


Monitoring Datasource Health

Key Metrics

Track datasource effectiveness:

Health Checklist

Lifecycle Management

  • Creation:
    • Set appropriate status (Active)
    • Clear owner assignment
    • Document purpose in description
  • Growth:
    • Monitor size and resource count
    • Keep metadata current
    • Update as content changes
  • Decline:
    • Mark as Archived if no longer used
    • Set Deprecated if replaced
    • Plan for eventual deletion
  • Retirement:
    • Final update to metadata
    • Archive for compliance
    • Delete if not needed

Best Practices

Clear Naming

Use descriptive, consistent names

Document Purpose

Write clear descriptions

Organize with Tags

Use tags for easy filtering

Assign Ownership

Clear responsibility for maintenance

Documentation

For each datasource, document:

Maintenance Schedule


Metadata Filtering in Agent Queries

Using allowed_vectors

When invoking an agent, you can restrict which resources are searched by passing allowed_vectors with metadata filters:

Filter Multiple Fields

Practical Example

Datasource Schema:
Resource Metadata:
Agent Call with Filter:
Agent Call with Multiple Filters:

Best Practices

Define Schema Early

Set metadata schema when creating the datasource

Keep It Simple

3-5 fields is usually enough for filtering

Use Boolean Sparingly

Good for flags (confidential, approved, published)

Document Fields

Explain what each metadata field means

Next Steps