Metadata Schema
What is a Metadata Schema?
A Metadata Schema defines a JSON structure that every resource in your datasource must follow. It’s like a contract:- Datasource with metadata schema:
- Field 1:
department(type:str, required) - Field 2:
priority(type:int, optional) - Field 3:
version(type:str, optional)
- Field 1:
- Every resource must provide:
department(string)priority(number) — optional, can be emptyversion(string) — optional, can be empty
Schema Field Definition
Each field in a metadata schema has:Example Metadata Schema
Creating a Metadata Schema
1
Go to Datasource
Navigate to the datasource you want to configure
2
Open Schema Configuration
Find Metadata Schema section in datasource settings
3
Define Fields
Add fields with name, type, and whether they’re optional
4
Set Required Fields
Mark critical fields as required (optional=false)
5
Save Schema
Schema is now enforced for all resources
Datasource Basic Properties
Name
Display name of the datasource
Description
Purpose and contents of the datasource
Type
Manual or Google Drive
Owner
Primary maintainer
Datasource Scope & Sharing
Primary Project
Project where datasource is created
Shared With Projects
Other projects that can access this datasource
Resource Count
Number of files, URLs, and content entries
Storage Size
Total size of all resources
Creating Datasource Metadata
Setting Up a New Datasource
1
Navigate to Datasources
Go to Datasources in your project
2
Create New Datasource
Click Create Datasource
3
Choose Type
Select Manual or Google Drive
4
Enter Basic Info
Fill in name and description
5
Configure Metadata
Set tags, visibility, and owner
6
Save
Datasource is created and ready
Metadata Fields Explained
Name
The display name for your datasource. Requirements:- Unique within your project
- Clear and descriptive
- 3-100 characters
- No special characters
Description
A summary of what the datasource contains. Purpose:- Help team members understand content
- Guide agent configuration
- Document coverage
- Explain use cases
- Main topics covered
- Update frequency
- Intended users/agents
- Maintenance status
- Key features
Type
The datasource category:Status
Track the datasource state:- Active
- Archived
- Deprecated
Currently in use
- Updated regularly
- Used by agents
- Primary reference
Tags
Categorize your datasources. Tag Examples:- Use consistent naming (lowercase, kebab-case)
- Limit to 5-10 tags per datasource
- Reuse existing tags for consistency
- Document tag meanings
Visibility
Control who can see and use the datasource:- Private
- Team
- Public
Only you can see and manage
- Not visible to others
- Full control
- Default for new datasources
Owner
The primary maintainer of the datasource. Responsibilities:- Keep content current
- Add/remove resources
- Update metadata
- Monitor quality
- Plan archival/deprecation
Resource Count
Shows how much content is in the datasource:
Example: Customer Support Datasource
Storage Size
Total size of all resources in the datasource. Considerations:- Impacts retrieval speed
- Affects storage quota
- Consider splitting if very large
- Archive old resources
- Archive old versions
- Remove duplicates
- Split by topic
- Target: under 200 MB
Last Updated
When content was last modified. Information:- Date of last change
- Who made the change
- What was updated
- Helps identify fresh content
- Verify content is current
- Plan updates
- Track maintenance
- Identify stale datasources
Agent Count
Number of agents using this datasource. Impact:- More agents = more usage
- Higher importance
- Wider impact of changes
- More careful updates needed
- Support Bot
- Email Agent
- Chat Assistant
- Knowledge Base Bot
- Escalation Agent
- FAQ Agent
- Billing Support
- Technical Support
Editing Datasource Metadata
Update Information
1
Find Datasource
Navigate to the datasource list
2
Open Settings
Click Edit or settings icon
3
Modify Fields
Update name, description, tags, status, etc.
4
Save Changes
Click Save to apply updates
5
Confirm
Changes are immediately available
What Can Be Changed
Editable:- Name (with limitations)
- Description
- Tags
- Status
- Owner
- Visibility
- Type (Manual vs Google Drive)
- Created date (system-generated)
- Resource count (updated automatically)
Using Metadata for Organization
Organizational Strategies
By Department:Naming Convention
Establish consistent naming:Datasource Organization Examples
Example 1: Support Team Structure
Example 2: Product Documentation
Monitoring Datasource Health
Key Metrics
Track datasource effectiveness:Health Checklist
Lifecycle Management
- Creation:
- Set appropriate status (Active)
- Clear owner assignment
- Document purpose in description
- Growth:
- Monitor size and resource count
- Keep metadata current
- Update as content changes
- Decline:
- Mark as Archived if no longer used
- Set Deprecated if replaced
- Plan for eventual deletion
- Retirement:
- Final update to metadata
- Archive for compliance
- Delete if not needed
Best Practices
Clear Naming
Use descriptive, consistent names
Document Purpose
Write clear descriptions
Organize with Tags
Use tags for easy filtering
Assign Ownership
Clear responsibility for maintenance
Documentation
For each datasource, document:Maintenance Schedule
Metadata Filtering in Agent Queries
Using allowed_vectors
When invoking an agent, you can restrict which resources are searched by passing allowed_vectors with metadata filters:Filter Multiple Fields
Practical Example
Datasource Schema:Best Practices
Define Schema Early
Set metadata schema when creating the datasource
Keep It Simple
3-5 fields is usually enough for filtering
Use Boolean Sparingly
Good for flags (confidential, approved, published)
Document Fields
Explain what each metadata field means
Next Steps
- Resource Metadata - Configure resource properties
- Resource Types - Learn about files, URLs, and content
- Cross-Project Sharing - Share datasources across projects
- Agent Configuration - Connect datasources to agents