Mission/open goal Description
I chose the Principal Data Analyst mission: identify the strongest evidence-backed product-quality risk cluster in Zava’s PromptathonDb.
The goal was to determine which product or product category should be prioritized for immediate intervention by combining:
- sales exposure,
- support-ticket burden,
- customer satisfaction,
- recurring complaint themes from chats and review documents,
- and semantic evidence from vector search.
The analysis ultimately narrowed the issue to SKU ZCPTM-SS-M-BW, Premium Short Sleeve Men's Top, where the strongest evidence pointed to a recurring smart-fabric wash/connectivity durability problem rather than a category-wide Premium product failure.
Harness and model
GitHub Copilot Chat in Agent mode, using the Auto model selection, with SQL MCP tools in GitHub Codespaces
Turn-by-turn journey
- Prompt:
Describe the available SQL MCP entities and summarize the role of each.
Agent response or action:
The agent used describe_entities to inspect the PromptathonDb entities and identified the sales, product, support, chat, document, customer, employee, and vector-search resources.
Result:
Confirmed that the Codespaces environment and SQL MCP connection were working.
- Prompt:
Inspect the detailed metadata and identify entity grain, keys, relationships, nullable fields, cardinalities, and duplication risks.
Agent response or action:
The named-entity metadata request failed, so the agent fell back to the full entity catalog. Its first assessment included inferred nullability and cardinality.
Result:
This exposed the need to verify plausible metadata conclusions with actual database queries rather than treating them as established facts.
- Prompt:
Verify row counts, key integrity, null rates, duplicates, and major relationship cardinalities with SQL MCP queries.
Agent response or action:
The agent retried several queries after connection and SQL errors, then verified counts and relationship coverage.
Result:
Primary identifiers were unique and non-null. Product linkage from SalesOrderLine was complete, while SalesOrder.CustomerId was mostly null and some support-to-SKU relationships were incomplete.
- Prompt:
Build the sales baseline using SalesOrderLine as the product grain and validate the revenue formula.
Agent response or action:
The agent initially joined order headers to lines in a way that inflated header totals. It detected the grain error, reran the comparison at order level, and used SalesOrderLine.LineTotal for product revenue.
Result:
Established a reliable commercial baseline and documented the difference between line-level revenue and SalesOrder header totals.
- Prompt:
Build the support baseline without multiplying ticket rows and use conservative product attribution.
Agent response or action:
The agent analyzed ticket volume, priority, status, satisfaction, related orders, and SKU coverage. It did not assign tickets from multi-line orders to a specific product without stronger evidence.
Result:
Premium Short Sleeve Men's Top had the largest attributed ticket count, while SKU ZCPTM-SS-M-BW had 9 tickets and average satisfaction of 1.67/5.
- Prompt:
Analyze SupportChat JSON and Docs for recurring complaint themes, while keeping chats and documents at their natural grains.
Agent response or action:
The agent parsed customer-authored chat messages and reviewed linked documents. It compared the leading candidate with other Premium SKUs.
Result:
The broad complaint picture narrowed to a specific pattern: smart-fabric connectivity and app-pairing failure after washing.
- Prompt:
Run find_similar_docs_by_doc_id using DocId 39 and compare the result with a general-quality complaint from another Premium product.
Agent response or action:
The agent used vector search and evaluated each neighbor rather than accepting the results uncritically.
Result:
The nearest documents formed a coherent multilingual cluster around wash-related connectivity failure. The metric was corrected from “similarity” to cosine distance, where lower values indicate closer matches.
- Prompt:
Act as a skeptical reviewer and try to disprove the leading hypothesis using ticket rates, sample sizes, competing products, duplicate evidence, and dataset skew.
Agent response or action:
The agent found two SKUs with higher raw ticket rates, but each had only two tickets. It also separated independent reviews from SupportChat documents that restated existing support records.
Result:
The conclusion remained strongest but was narrowed from a broad product-quality crisis to a specific SKU-level wash/connectivity durability problem.
- Prompt:
Check evidence independence and whether the target SKU was unusually represented in the seeded dataset.
Agent response or action:
The agent counted distinct tickets, chats, reviews, sources, orders, and customers, and compared document volume across SKUs.
Result:
The target SKU had 6 independent review documents and 8 derivative SupportChat documents. It also had unusually heavy document seeding, which increased confidence in the existence of the pattern but reduced confidence in claims about unbiased prevalence.
Completion
Bonus work
Beyond the core mission, I added several analytical safeguards:
- Verified metadata assumptions with actual SQL MCP queries instead of relying on inferred nullability or cardinality.
- Detected and corrected a join-grain error that inflated SalesOrder header totals.
- Used conservative ticket attribution for multi-line orders and missing SKU relationships.
- Compared ticket rates as well as raw ticket counts.
- Challenged the leading conclusion with alternative SKUs and sample-size checks.
- Distinguished independent review evidence from derivative SupportChat documents.
- Evaluated vector-search neighbors critically and corrected the metric label to cosine distance.
- Checked for duplicate evidence and unusually heavy document seeding.
- Created two supporting Markdown artifacts in my fork:
- Data Analyst Executive Brief
- Prompt Journey
Mission/open goal Description
I chose the Principal Data Analyst mission: identify the strongest evidence-backed product-quality risk cluster in Zava’s PromptathonDb.
The goal was to determine which product or product category should be prioritized for immediate intervention by combining:
The analysis ultimately narrowed the issue to SKU ZCPTM-SS-M-BW, Premium Short Sleeve Men's Top, where the strongest evidence pointed to a recurring smart-fabric wash/connectivity durability problem rather than a category-wide Premium product failure.
Harness and model
GitHub Copilot Chat in Agent mode, using the Auto model selection, with SQL MCP tools in GitHub Codespaces
Turn-by-turn journey
Describe the available SQL MCP entities and summarize the role of each.
Agent response or action:
The agent used describe_entities to inspect the PromptathonDb entities and identified the sales, product, support, chat, document, customer, employee, and vector-search resources.
Result:
Confirmed that the Codespaces environment and SQL MCP connection were working.
Inspect the detailed metadata and identify entity grain, keys, relationships, nullable fields, cardinalities, and duplication risks.
Agent response or action:
The named-entity metadata request failed, so the agent fell back to the full entity catalog. Its first assessment included inferred nullability and cardinality.
Result:
This exposed the need to verify plausible metadata conclusions with actual database queries rather than treating them as established facts.
Verify row counts, key integrity, null rates, duplicates, and major relationship cardinalities with SQL MCP queries.
Agent response or action:
The agent retried several queries after connection and SQL errors, then verified counts and relationship coverage.
Result:
Primary identifiers were unique and non-null. Product linkage from SalesOrderLine was complete, while SalesOrder.CustomerId was mostly null and some support-to-SKU relationships were incomplete.
Build the sales baseline using SalesOrderLine as the product grain and validate the revenue formula.
Agent response or action:
The agent initially joined order headers to lines in a way that inflated header totals. It detected the grain error, reran the comparison at order level, and used SalesOrderLine.LineTotal for product revenue.
Result:
Established a reliable commercial baseline and documented the difference between line-level revenue and SalesOrder header totals.
Build the support baseline without multiplying ticket rows and use conservative product attribution.
Agent response or action:
The agent analyzed ticket volume, priority, status, satisfaction, related orders, and SKU coverage. It did not assign tickets from multi-line orders to a specific product without stronger evidence.
Result:
Premium Short Sleeve Men's Top had the largest attributed ticket count, while SKU ZCPTM-SS-M-BW had 9 tickets and average satisfaction of 1.67/5.
Analyze SupportChat JSON and Docs for recurring complaint themes, while keeping chats and documents at their natural grains.
Agent response or action:
The agent parsed customer-authored chat messages and reviewed linked documents. It compared the leading candidate with other Premium SKUs.
Result:
The broad complaint picture narrowed to a specific pattern: smart-fabric connectivity and app-pairing failure after washing.
Run find_similar_docs_by_doc_id using DocId 39 and compare the result with a general-quality complaint from another Premium product.
Agent response or action:
The agent used vector search and evaluated each neighbor rather than accepting the results uncritically.
Result:
The nearest documents formed a coherent multilingual cluster around wash-related connectivity failure. The metric was corrected from “similarity” to cosine distance, where lower values indicate closer matches.
Act as a skeptical reviewer and try to disprove the leading hypothesis using ticket rates, sample sizes, competing products, duplicate evidence, and dataset skew.
Agent response or action:
The agent found two SKUs with higher raw ticket rates, but each had only two tickets. It also separated independent reviews from SupportChat documents that restated existing support records.
Result:
The conclusion remained strongest but was narrowed from a broad product-quality crisis to a specific SKU-level wash/connectivity durability problem.
Check evidence independence and whether the target SKU was unusually represented in the seeded dataset.
Agent response or action:
The agent counted distinct tickets, chats, reviews, sources, orders, and customers, and compared document volume across SKUs.
Result:
The target SKU had 6 independent review documents and 8 derivative SupportChat documents. It also had unusually heavy document seeding, which increased confidence in the existence of the pattern but reduced confidence in claims about unbiased prevalence.
Completion
Bonus work
Beyond the core mission, I added several analytical safeguards: