Delivery is fast. Now, shopping is too.
An AI-powered conversational commerce layer built for HackOn with Amazon — Season 6.0
Quick-commerce delivery is a solved problem—orders arrive in 10 minutes. Shopping itself is not. Customers arrive with an immediate, high-stress need but still have to endure friction: search, compare, build a cart, and spend minutes deciding.
The friction lives between the need and the checkout.
| What customers do today | What they actually want |
|---|---|
| Type "milk" → Scroll 10 brands → Add to cart → Type "bread"... | "Breakfast for 2" → Auto-Cart → Swipe to Order |
Amazon Now completely reimagines urgent shopping. You tell Amazon Now your outcome (via text, voice, or photo), and the Now Agent handles the cognitive load of picking the best products, checking your budget, substituting out-of-stock items, and explaining why it chose what it did.
"kal subah breakfast, 2 log, under ₹300" — or a photo of your empty fridge — becomes a complete, budget-checked cart. One swipe to buy.
- Multimodal Intent: Snap a photo of an empty fridge, a handwritten list, a product box, or a recipe; the agent reads it, extracts the sub-needs, and builds the cart.
- Recipe / Occasion-to-Cart: "Paneer butter masala for 4" → ingredients automatically scaled to servings, smartly excluding staples you likely already own (salt, oil).
- Automated Budget Rebalancing: Over budget? The agent re-optimizes in real-time, swapping expensive items for cheaper equivalents ("saved ₹40 on milk"), and lets you easily revert.
- Contextual Memory: A fully integrated messaging UI that remembers the entire conversation. If a request is too broad (e.g., "iPhone"), the agent gracefully asks follow-up questions instead of guessing.
- Polite Substitutions: If your requested item is out of stock, the agent won't silently swap it. It actively pauses and asks, "Amul Butter is out of stock. Would you like me to find a substitute?"
- Seamless Fallbacks: The system auto-upgrades "Quick" mode searches into "Chat" mode seamlessly if it absolutely needs human clarification.
- Google Calendar Sync: Securely syncs with your Google Calendar to proactively suggest carts for upcoming events (e.g., Birthdays, Meetings, Travel).
- Consumption-Rate Prediction: Models how fast you use essentials ("2-day milk cycle → out Thursday").
- Hyper-Local Context: Dynamically suggests carts based on local weather (e.g., hydration packs during heatwaves) and Indian festivals.
- Extreme Scenarios: One-tap SOS chips for intense emergencies like Cut Finger (curates first-aid, dettol, band-aids), Burned Cooking, or Severe Cramps.
- Done in Seconds: Bypasses the entire shopping funnel to provide exactly what is needed instantly.
- Explicit Trust Badges: Items feature dynamic badges (🏆 Top Match, 💰 Best Value, 🌿 Vegan) driven by AI confidence scores.
- Swipe-to-Order: Replaces the boring "Buy Now" button with a highly tactile, satisfying swipe slider.
- Live Delivery Radar: Post-checkout features a glowing, pulsing radar animation and a live 10-minute countdown timer, proving the "Delivery is Fast" claim visually.
Amazon Now does not rebuild Amazon's storefront or logistics; it builds the intelligence layer on top.
- Frontend: React + TypeScript + Vite + Tailwind CSS (Mobile-first, Premium Amazon Aesthetic, Sleek Dark Mode)
- Backend: Node.js + Express (AWS Lambda-ready architecture)
- Database: AWS DynamoDB (Full product catalog with ratings, Prime badges, delivery times, and order history)
- AI / LLM Reasoning: AWS Bedrock (Nova Lite) powering reasoning, vision, and chat orchestration.
- Embeddings: AWS Bedrock (Titan Text Embeddings V2)
- Search: In-Memory Vector Search (Cosine Similarity) with intelligent caching.
- Parse Intent: Converts raw input (Text, voice, or image) into a structured goal + constraints using LLM.
- Context Retrieval: Injects user history, learned preferences (Avoid/Prefer), and DynamoDB profile data.
- Decompose: Recipes/occasions are intelligently broken down into sub-needs and scaled to servings.
- Vector Search: Performs semantic cosine-similarity searches against the catalog embeddings.
- Assemble & Evaluate: LLM evaluates candidates. If out of stock, asks for substitutes via Chat. If details are missing, triggers clarifying questions.
- Rebalance: Enforces budget deterministically by swapping to cheaper equivalents in the same category.
- Explain: Outputs a cart with dynamic Trust Badges, confidence scores, and nudges.
- Node.js 18+
- AWS account + credentials (for DynamoDB and Bedrock)
- Amazon Bedrock access enabled for:
us.amazon.nova-lite-v1:0amazon.titan-embed-text-v2:0
# 1. Install dependencies
cd backend && npm install
cd ../frontend && npm install
# 2. Configure Environment (backend/.env)
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=<your-aws-access-key>
AWS_SECRET_ACCESS_KEY=<your-aws-secret-key>
GOOGLE_CLIENT_ID=<your-google-client-id>
GOOGLE_CLIENT_SECRET=<your-google-client-secret>
GOOGLE_REDIRECT_URI=http://localhost:4001/api/auth/google/callback
FRONTEND_URL=http://localhost:5173
# 3. Seed the Database
cd backend
npx tsx scripts/migrate-dynamodb.ts # Migrates Amazon Metadata to DynamoDB
# 4. Start the Application
npm run dev # Runs backend on port 4001
cd ../frontend && npm run dev # Runs frontend on port 5173- Voice-First Urgent Shopping: "I have guests in an hour."
- Population-Scale Proactivity: Carts that appear before the customer even opens the app.
- Confidence as a Moat: Explainable carts that earn trust on every purchase.
From need to done, in seconds — for millions.
Made with ❤️ for HackOn with Amazon — Season 6.0