Redis Alternatives Analysis for AgentPortalBFF¶

Executive Summary¶

Based on analysis of the AgentPortalBFF codebase, this document evaluates 7 alternatives to replace Redis for caching, search, and JSON document storage. The current Redis implementation uses RedisJSON for document storage and RediSearch for full-text search with complex filtering.

Quick Decision Matrix¶

Recommended Solution: Typesense Cloud - offers the best balance of features, performance, and cost for your use case.

Current Redis Usage Analysis¶

Redis Features in Use¶

The AgentPortalBFF currently uses Redis with the following components:

graph TB
    A[AgentPortalBFF] --> B[RedisJSON]
    A --> C[RediSearch]
    A --> D[Standard Redis]

    B --> E[JSON Document Storage]
    B --> F[JSONPath Queries]

    C --> G[Full-text Search]
    C --> H[Multiple Indexes]
    C --> I[Complex Filtering]

    D --> J[Key-Value Caching]
    D --> K[TTL/Expiration]

    H --> L[idx-bff-consultant]
    H --> M[idx-bff-request]
    H --> N[idx-map-locations]

Critical Operations Inventory¶

1. JSON Document Operations
2. json.SetAsync() - Store complex JSON documents
3. json.GetAsync() - Retrieve with JSONPath selectors
4. json.ArrAppendAsync() - Append to JSON arrays

5. json.TypeAsync() - Check field existence

6. Search Operations

7. FT.CREATE - Create search indexes
8. FT.SEARCH - Complex queries with filtering
9. Full-text search on flattened content
10. Geo-spatial queries

11. Numeric range filtering

12. Index Schemas

    FT.CREATE idx-bff-consultant ON JSON 
    PREFIX 1 BFF:consultant: 
    SCHEMA 
        $.ContactDto.Fullname AS Fullname TEXT NOSTEM 
        $.ProfileLevel2.MatchingType AS ProfileMatchingType NUMERIC 
        $.EntityId AS EntityId TAG 
        $.Filters.Territory AS Territory TAG 
        $.Filters.Status as Status TAG 
        $.Filters.MapGeoLocation AS MapGeoLocation GEO 
        $.Matches[*].AverageSimilarity AS AverageSimilarity NUMERIC SORTABLE
    

13. Data Types Used

14. Profile documents (Level0-3)
15. Request documents
16. Match results with nested arrays
17. Map location data with geo-coordinates
18. Filter metadata

Alternative Solutions¶

1. Typesense Cloud ⭐⭐⭐⭐⭐¶

Overview: Modern search engine designed for speed and ease of use. Perfect match for your documented Redis-to-Typesense migration.

Key Features: - Native JSON document support - Excellent full-text search with typo tolerance - Built-in faceting and filtering - Geo-search capabilities - Keyword highlighting out-of-the-box - Fast pagination with search-after cursors

Azure Integration: - Typesense Cloud (managed service) - Can deploy on Azure VMs if needed - REST API integration

Code Example:

// Replace Redis JSON operations
var document = new {
    id = profileId.ToString(),
    fullname = profile.ContactDto.Fullname,
    status = profile.Filters.Status,
    location = new { lat = lat, lon = lon }
};

await typesenseClient.Collections["consultants"]
    .Documents.Create(document);

// Replace FT.SEARCH operations
var searchParams = new SearchParameters {
    Q = "software engineer",
    QueryBy = "fullname,flattened_content",
    FilterBy = "status:=active",
    Facet_by = "territory,office_location",
    Highlight_full_fields = "fullname",
    Per_page = 20,
    Page = pageNumber
};

Advantages: - 83% cost reduction (as noted in your docs) - Built-in keyword highlighting - Excellent .NET SDK - Minimal code changes needed - Real-time indexing - Horizontal scaling

Disadvantages: - Newer technology (less enterprise adoption) - Limited complex aggregations - No built-in TTL (must implement in application)

Migration Effort: Low - Direct replacement for most Redis operations

Cost Estimate (50GB): - Typesense Cloud: $30-50/month - Additional bandwidth: $10-20/month

2. Azure AI Search ⭐⭐⭐⭐¶

Overview: Fully managed search service with powerful AI capabilities and excellent Azure integration.

Key Features: - Rich JSON document support - Advanced full-text search - Built-in faceting and filtering - Geo-spatial search - Semantic search capabilities - Custom scoring profiles

Code Example:

// Document indexing
var consultant = new SearchDocument() {
    ["id"] = profileId.ToString(),
    ["fullname"] = profile.ContactDto.Fullname,
    ["status"] = profile.Filters.Status,
    ["location"] = GeographyPoint.Create(lat, lon),
    ["content"] = profile.FlattenedContent
};

await searchClient.UploadDocumentsAsync([consultant]);

// Search with highlighting
var searchOptions = new SearchOptions {
    Filter = "status eq 'active'",
    Facets = { "territory", "office_location" },
    HighlightFields = { "fullname", "content" },
    Size = 20,
    Skip = pageNumber * 20
};

var response = await searchClient.SearchAsync<SearchDocument>(
    "software engineer", searchOptions);

Advantages: - Native Azure service - Excellent .NET SDK - Built-in highlighting and faceting - AI-powered features - Strong security integration - Automatic scaling

Disadvantages: - More expensive than Typesense - Complex pricing model - Over-engineered for simple use cases - Learning curve for advanced features

Migration Effort: Medium - Requires schema redesign and query adaptation

Cost Estimate (50GB): - Standard S2 tier: $100-200/month - Storage and queries included

3. Elasticsearch on Azure ⭐⭐⭐⭐¶

Overview: Industry-standard search engine with comprehensive features and Azure marketplace availability.

Key Features: - Excellent JSON document support - Powerful query DSL - Advanced aggregations - Geo-spatial capabilities - Rich ecosystem of tools - Machine learning features

Azure Deployment Options: - Elastic Cloud on Azure Marketplace - Self-managed on Azure VMs - Azure partner offering

Code Example:

// Document indexing
var indexRequest = new IndexRequest<ProfileDocument>("consultants") {
    Document = new ProfileDocument {
        Id = profileId,
        Fullname = profile.ContactDto.Fullname,
        Status = profile.Filters.Status,
        Location = new GeoLocation(lat, lon),
        Content = profile.FlattenedContent
    }
};

await elasticClient.IndexAsync(indexRequest);

// Complex search query
var searchRequest = new SearchRequest<ProfileDocument> {
    Query = new BoolQuery {
        Must = new QueryContainer[] {
            new MatchQuery { Field = "content", Query = "software engineer" }
        },
        Filter = new QueryContainer[] {
            new TermQuery { Field = "status", Value = "active" }
        }
    },
    Aggregations = new Dictionary<string, AggregationContainer> {
        ["territories"] = new TermsAggregation("territories") { Field = "territory" }
    },
    Highlight = new Highlight {
        Fields = new Dictionary<string, IHighlightField> {
            ["fullname"] = new HighlightField(),
            ["content"] = new HighlightField()
        }
    },
    Size = 20,
    From = pageNumber * 20
};

Advantages: - Battle-tested technology - Comprehensive feature set - Excellent .NET client - Strong community support - Advanced analytics capabilities - Flexible deployment options

Disadvantages: - Complex configuration - Higher operational overhead - Memory intensive - Steep learning curve - Licensing considerations

Migration Effort: Medium - Well-documented migration patterns from Redis

Cost Estimate (50GB): - Elastic Cloud: $80-150/month - Self-managed on Azure: $50-100/month + VM costs

4. Azure Cosmos DB ⭐⭐⭐¶

Overview: Microsoft's globally distributed NoSQL database with multiple API support.

Key Features: - Native JSON document storage - Global distribution - Multiple consistency levels - Integrated search (limited) - Automatic scaling - Strong SLA guarantees

Code Example:

// Document storage (similar to RedisJSON)
await container.CreateItemAsync(profileDocument, 
    new PartitionKey(profileDocument.TenantId));

// Querying (limited search capabilities)
var query = "SELECT * FROM c WHERE CONTAINS(c.content, 'software') 
            AND c.status = 'active'";
var results = container.GetItemQueryIterator<ProfileDocument>(query);

Advantages: - Native Azure service - Excellent global distribution - Strong consistency guarantees - JSON-native - Automatic scaling - Multi-model support

Disadvantages: - Limited search capabilities - Expensive for search workloads - No built-in highlighting - Complex pricing model - RU-based costing can be unpredictable

Migration Effort: High - Requires significant architectural changes

Cost Estimate (50GB): - Provisioned throughput: $200-400/month - Serverless: Variable, potentially higher

5. Azure Database for PostgreSQL ⭐⭐⭐¶

Overview: Managed PostgreSQL with excellent JSON support and full-text search capabilities.

Key Features: - Advanced JSON/JSONB support - Full-text search with GIN indexes - Trigram similarity search - Geographic extensions (PostGIS) - Strong consistency - ACID compliance

Code Example:

// JSON document storage
var sql = @"INSERT INTO consultants (id, document) 
           VALUES (@id, @document::jsonb)";
await connection.ExecuteAsync(sql, new { 
    id = profileId, 
    document = JsonSerializer.Serialize(profile) 
});

// Full-text search
var searchSql = @"
    SELECT id, document, 
           ts_headline(document->>'content', query) as highlight
    FROM consultants, 
         plainto_tsquery('english', @searchTerm) query
    WHERE to_tsvector('english', document->>'content') @@ query
    AND document->>'status' = @status
    ORDER BY ts_rank(to_tsvector('english', document->>'content'), query) DESC
    LIMIT @limit OFFSET @offset";

Advantages: - SQL familiarity - Strong JSON support - Cost-effective - ACID compliance - Mature ecosystem - Excellent .NET support

Disadvantages: - Limited highlighting features - Complex full-text search setup - Manual index management - Less search-optimized than dedicated solutions

Migration Effort: High - Requires significant schema and query redesign

Cost Estimate (50GB): - General Purpose: $50-100/month - Additional storage: $20-30/month

6. Azure SQL Database with JSON ⭐⭐¶

Overview: Traditional relational database with JSON support and full-text search capabilities.

Key Features: - JSON support in SQL Server 2016+ - Full-text indexing - Familiar SQL interface - Strong enterprise features - Integration with existing SQL skills

Code Example:

// JSON storage
var sql = @"INSERT INTO consultants (id, json_data) 
           VALUES (@id, @jsonData)";
await connection.ExecuteAsync(sql, new { 
    id = profileId, 
    jsonData = JsonSerializer.Serialize(profile) 
});

// Search with JSON functions
var searchSql = @"
    SELECT id, json_data
    FROM consultants
    WHERE CONTAINS(json_data, @searchTerm)
    AND JSON_VALUE(json_data, '$.status') = @status
    ORDER BY id
    OFFSET @offset ROWS FETCH NEXT @limit ROWS ONLY";

Advantages: - SQL expertise available - Enterprise-grade features - Strong security - Integration with existing tools - Predictable costs

Disadvantages: - Limited JSON query capabilities - Poor search performance - No built-in highlighting - Relational mindset clash - Expensive for document workloads

Migration Effort: High - Fundamental architecture change required

Cost Estimate (50GB): - General Purpose S4: $100-200/month

7. MongoDB Atlas on Azure ⭐⭐⭐¶

Overview: Fully managed MongoDB service available on Azure with search capabilities.

Key Features: - Native document storage - Atlas Search (Elasticsearch-based) - Flexible schema - Horizontal scaling - Rich query language - Change streams

Code Example:

// Document storage
await collection.InsertOneAsync(profileDocument);

// Search with Atlas Search
var pipeline = new BsonDocument[]
{
    new BsonDocument("$search", new BsonDocument
    {
        { "index", "consultants_search" },
        { "text", new BsonDocument
            {
                { "query", "software engineer" },
                { "path", new BsonArray { "fullname", "content" } }
            }
        },
        { "highlight", new BsonDocument("path", "content") }
    }),
    new BsonDocument("$match", new BsonDocument("status", "active")),
    new BsonDocument("$limit", 20),
    new BsonDocument("$skip", pageNumber * 20)
};

var results = await collection.AggregateAsync<ProfileDocument>(pipeline);

Advantages: - Document-native approach - Flexible schema evolution - Good search capabilities (Atlas Search) - Strong .NET driver - Familiar to NoSQL developers

Disadvantages: - Additional vendor dependency - Atlas Search requires higher tiers - More complex than specialized search solutions - Operational complexity

Migration Effort: Medium - Document model fits well but requires query changes

Cost Estimate (50GB): - M30 cluster with Atlas Search: $150-300/month

Detailed Feature Comparison¶

JSON Document Support¶

Search Capabilities¶

Operational Considerations¶

Architecture Diagrams¶

Current Redis Architecture¶

graph TB
    subgraph "AgentPortalBFF"
        A[Functions] --> B[RedisApplicationService]
        A --> C[MatchingRedisService]
        B --> D[RedisAdapter]
        C --> D
    end

    subgraph "Redis Cluster"
        D --> E[RedisJSON Module]
        D --> F[RediSearch Module]
        E --> G[(JSON Documents)]
        F --> H[Search Indexes]
    end

    subgraph "Data Types"
        G --> I[Profile Documents]
        G --> J[Request Documents]
        G --> K[Match Results]
        H --> L[Consultant Index]
        H --> M[Request Index]
        H --> N[Map Location Index]
    end

Proposed Typesense Architecture¶

graph TB
    subgraph "AgentPortalBFF"
        A[Functions] --> B[TypesenseApplicationService]
        B --> C[TypesenseAdapter]
    end

    subgraph "Typesense Cloud"
        C --> D[Consultant Collection]
        C --> E[Request Collection]
        C --> F[Map Location Collection]
    end

    subgraph "Features"
        D --> G[Full-text Search]
        D --> H[Faceted Filtering]
        D --> I[Geo Search]
        E --> G
        E --> H
        F --> I
        G --> J[Keyword Highlighting]
        H --> K[Real-time Filtering]
        I --> L[Location-based Search]
    end

Proposed Azure AI Search Architecture¶

graph TB
    subgraph "AgentPortalBFF"
        A[Functions] --> B[SearchApplicationService]
        B --> C[SearchServiceClient]
    end

    subgraph "Azure AI Search"
        C --> D[Consultant Index]
        C --> E[Request Index]
        C --> F[Map Location Index]
    end

    subgraph "Azure Integration"
        D --> G[Cognitive Services]
        E --> G
        F --> G
        G --> H[AI Enrichment]
        G --> I[Semantic Search]
        D --> J[Built-in Scaling]
        E --> J
        F --> J
    end

Migration Strategies¶

1. Typesense Migration (Recommended)¶

Phase 1: Parallel Implementation (2-3 weeks) - Set up Typesense Cloud instance - Create collections matching Redis indexes - Implement TypesenseAdapter alongside RedisAdapter - Run dual-write pattern for new data

Phase 2: Data Migration (1 week) - Export existing Redis data - Transform and import into Typesense - Validate data integrity - Performance testing

Phase 3: Cutover (1 week) - Feature flag to switch reads to Typesense - Monitor performance and errors - Gradual rollout to all endpoints - Remove Redis dependencies

Code Changes Required:

// Before (Redis)
var json = redis.JSON();
await json.SetAsync(key, "$", document, serializerOptions);

var ft = redis.FT();
var results = await ft.SearchAsync(index, query.Limit(offset, limit));

// After (Typesense)
await typesenseClient.Collections[collectionName]
    .Documents.Create(document);

var searchParams = new SearchParameters {
    Q = searchTerm,
    QueryBy = "searchable_fields",
    FilterBy = filters,
    Per_page = limit,
    Page = pageNumber
};
var results = await typesenseClient.Collections[collectionName]
    .Documents.Search(searchParams);

2. Azure AI Search Migration¶

Phase 1: Schema Design (2 weeks) - Design search indexes - Set up indexers for data ingestion - Configure skillsets if needed - Test search capabilities

Phase 2: Service Implementation (3 weeks) - Implement SearchApplicationService - Adapt existing queries to Azure AI Search syntax - Handle result transformation - Implement faceting and filtering

Phase 3: Migration & Cutover (2 weeks) - Migrate data to search indexes - Parallel testing - Performance optimization - Production cutover

3. Elasticsearch Migration¶

Phase 1: Infrastructure Setup (2 weeks) - Set up Elastic Cloud on Azure - Configure cluster settings - Design index mappings - Set up monitoring

Phase 2: Application Integration (3 weeks) - Implement ElasticsearchAdapter - Adapt complex queries to Elasticsearch DSL - Implement aggregations for faceting - Handle highlighting and search features

Phase 3: Data Migration (2 weeks) - Use Elasticsearch bulk APIs for data import - Reindex operations - Performance tuning - Production deployment

Cost Analysis Deep Dive¶

50GB Storage + Search Workload Assumptions¶

• Documents: ~1M consultant profiles, 100K requests
• Search QPS: ~100 queries/second peak
• Index Updates: ~1000 documents/hour
• Retention: 2 years

Detailed Cost Breakdown¶

Cost Savings vs Current Redis: Typesense offers 83% cost reduction as documented in your migration notes.

Recommendations¶

Primary Recommendation: Typesense Cloud¶

Why Typesense is the best fit:

1. Perfect Feature Match: Covers all Redis functionality you're using
2. Cost Effective: 83% cost reduction from current Redis implementation
3. Low Migration Risk: Minimal code changes required
4. Better Features: Built-in highlighting, typo tolerance, better faceting
5. Cloud Native: Fully managed with excellent uptime
6. Future Proof: Modern architecture designed for scale

Secondary Options¶

Azure AI Search - If you need enterprise features and have budget flexibility Elasticsearch - If you want battle-tested technology and don't mind complexity

Implementation Approach¶

1. Start with Typesense pilot (4 weeks)
2. Implement one collection (consultants)
3. Validate performance and features

4. Compare with Redis performance

5. Full migration if pilot succeeds (6 weeks)

6. Migrate all collections
7. Implement dual-write pattern

8. Gradual cutover with rollback plan

9. Optimization phase (2 weeks)

10. Performance tuning
11. Cost optimization
12. Monitoring setup

Risk Mitigation¶

• Parallel running: Keep Redis running during migration
• Feature flags: Control traffic routing between systems
• Rollback plan: Can quickly revert to Redis if issues occur
• Performance monitoring: Comprehensive metrics to track migration success

Next Steps¶

1. Stakeholder Review: Review this analysis with team and stakeholders
2. Pilot Decision: Choose solution for pilot implementation
3. Resource Planning: Allocate development resources for migration
4. Timeline Planning: Create detailed migration timeline
5. Success Metrics: Define KPIs for migration success

This analysis is based on current AgentPortalBFF Redis usage patterns and 50GB storage requirements. Costs are estimates and should be validated with current vendor pricing.