Time To Say Dubai – Multilingual GenAI Chatbot & Data Pipeline via AWS Lambda
AWS Cloud Operations Competency
Client: Time To Say Dubai
ACE Opportunity ID: 06791808
Domain: AWS Cloud Operations
Pattern: Governance, Compliance, Observability, and Operations with AWS CloudOps across Multi-Account Setup
1. Business Problem
Time2SayDubai provides travel, business setup, and job consultancy in Dubai. The company needed a reliable, scalable system to ingest job and event data dynamically, support multilingual conversational customer interactions, and deliver personalized recommendations and information. Existing manual scraping, translation, and data workflows were slow, error-prone, and could not support real-time user queries or GenAI-driven FAQs.
2. Project Objective
- Automate ingestion of job and event data scraped via Apify, reducing delay and manual work.
- Build a conversational AI chatbot combining structured data (jobs/events) and generative AI for general queries.
- Support multiple languages with translation and sentiment detection.
- Seamlessly integrate backend with frontend (web & mobile) using AWS Amplify.
- Achieve scalable, serverless design using AWS Lambda to minimize infrastructure management and cost.
3. Business Problem and Traditional Approach
Traditional Approach Challenges:
- Manual Data Synchronization: Data was scraped and moved manually or through semi-automated scripts, causing lag and inconsistencies.
- Language Barriers: Users speaking different languages had poor experience due to lack of translation or context.
- Fragmented Systems: Separate tools for scraping, storage, chat, generative AI lacking unified orchestration.
- Limited Responsiveness: Latency between scraping runs and availability of updated data. Generative FAQ responses often generic due to insufficient context or old data.
- High Operational Overhead: Maintenance of servers or always-on services (for translation, scraping, etc.) added cost and complexity.
4. AWS Lambda-Centric Solution Architecture
Time2SayDubai implemented a serverless, event-driven architecture centered on AWS Lambda functions to support data ingestion, conversational AI, multilingual support, and frontend integration. Key components:
- Webhook Lambda (Apify → SNS Trigger): Receives webhooks from Apify after a scraping run, validates payload, and publishes metadata (Run ID + metadata) to Amazon SNS.
- Data Fetcher & Storage Lambda (SNS → RDS Updater): Triggered by SNS; uses Run ID to fetch raw scraped data via Apify API; transforms it into normalized schema; persists into Amazon Aurora (MySQL-compatible RDS) into tables: job_listings, upcoming_events, user_bookings.
- Lex Backend Lambda (Conversational Orchestrator): Activated when a user query comes in via Amazon Lex; determines intent (Event Info / Job Search / FAQ); for structured queries hits RDS; for general FAQ uses Amazon Bedrock (Claude or another LLM); composes unified responses.
- Translation & NLP Connector Lambda: Detects input language and sentiment via Amazon Comprehend, uses Amazon Translate to translate inputs/outputs as needed, ensuring user queries and responses are presented in user’s preferred language.
- Amplify Connector Lambda (Frontend Integration): Acts as secure gateway between frontend (web/mobile via AWS Amplify) and backend chat services; conveys session context, user preferences, and handles UI-initiated flows like event booking, subscription, etc.
Supporting services: SNS for decoupling, API Gateway for exposing endpoints (if needed), EventBridge for scheduling or lifecycle events, CloudWatch for logs and metrics.
All components are deployed using AWS SAM for Infrastructure as Code, versioning, and CI/CD.
5. Quantitative Business Impact Metrics
| Metric | Before Lambda Implementation | After Lambda Implementation |
| Data update latency (scraped jobs/events to availability) | Hours or delayed until manual/cron trigger | Reduced to minutes via Apify webhook → SNS → Lambda pipeline |
| Multilingual user interactions | Limited or delayed translation; poor sentiment understanding | Support for multiple languages; sentiment detection; real-time translation in/out |
| Manual effort in data processing / ingestion | Significant manual scripting or oversight | Automated pipeline with minimal human intervention |
| Operational cost (servers, manual processes) | Higher fixed cost and overhead | Pay-per-invocation serverless cost; reduced operational overhead |
| Conversational response consistency and coverage | FAQ/general queries often disconnected from structured data; slower responses | Lex + Bedrock combo yields richer responses; structured data backed job/event info returned immediately |
| Scalability & maintenance | Scaling required manual resource provisioning; risk of system degradation | Lambda automatically scales; maintenance overhead lowered significantly |
6. Business-Led Transformation Using Generative AI & Serverless Lambda
By combining structured data (jobs, events) with generative AI, Time2SayDubai transformed its customer-facing experience. Users can now query job/event listings, ask general FAQs, in their own language, and get answers that draw from both the latest scraped data + GenAI responses.
Automation reduced delay, improved data freshness, increased user satisfaction, and allowed the business to focus on improving features rather than managing infrastructure. The serverless architecture enabled faster deployment of new conversational flows, languages, and integrations.
7. Deployment and Production Readiness on AWS
- Deployment Environment: Fully serverless AWS stack; Lambdas, SNS, RDS Aurora, Lex, Bedrock, Translate, Comprehend.
- Production Status: Solution deployed and serving real users; chat, recommendations, event/job listing flows in production.
- Scalability Strategy: SNS + Lambda for parallel ingestion; Lambda concurrency settings; autoscaling of RDS as needed.
- Security & Data Privacy: IAM roles with least privilege; encrypted storage; secure front-end integration; session management.
- Observability and Monitoring: CloudWatch monitoring of Lambda duration, error rates; logs for tracking scraping runs, Lex usage, translation success; metrics for user engagement, latency.
- Infrastructure as Code: AWS SAM used for versioned, repeatable deployments across dev/staging/prod.
8. Lambda Best Practices & Lessons Incorporated
- Modular, single-responsibility Lambdas (one for ingestion, one for conversation, etc.).
- Decoupled architecture using SNS for event-driven workflows, minimizing tight coupling.
- Fallback paths: structured queries vs generative ones when necessary.
- Prompt tuning & versioning for GenAI responses to reduce hallucination and improve coherence.
- Efficient memory/timeouts for Lambdas to balance cost vs performance.
- Monitoring & alerting for risk detection, including translation errors, Lex fallback usage, failed scraping fetches.
9. Conclusion
Time2SayDubai’s adoption of an AWS Lambda-centric, serverless architecture enabled the transformation of its platform into a real-time, multilingual, and AI-assisted consultancy chatbot solution. The result was fresher job/event data availability, higher quality conversational responses combining structured and generative data, multilingual support, reduced operational overhead, and enhanced scalability.
By shifting to event-driven workflows and leveraging managed AI services (Lex, Bedrock, Translate, Comprehend), Time2SayDubai reduced manual effort, improved user engagement, and positioned itself for rapid iteration and growth all while maintaining cost efficiency and reliability.
Solution Architecture:
