diff --git a/02_activities/assignments/DC_Cohort/Assignment1.md b/02_activities/assignments/DC_Cohort/Assignment1.md index d5ff55ce4..7e5b81873 100644 --- a/02_activities/assignments/DC_Cohort/Assignment1.md +++ b/02_activities/assignments/DC_Cohort/Assignment1.md @@ -108,6 +108,11 @@ Please do not pick the exact same tables that I have already diagrammed. For exa *** +## Response to section 1: The Logical data model for customer and customer_purchases + + + +## Prompt 2 + + +## Prompt 3 ``` -Your answer... +The bookstore could manage customer addresses in two ways, depending on whether it needs to keep previous addresses. These two ways include Type 1, which overwrites the previous address, and Type 2, which keeps a history of address changes. Both designs assume that each customer has one current address. + +For Type 1: overwrite the previous address. We will design the customer address table to include the customer id, address, city, province, postal code, country, and updated_at columns. The updated_at field will be populated when a customer changes their address. This update can only be done or replaced in the store, since each customer has one row linked to the customer table through customer id. For example, if a customer moves from 12 King Street to 45 Queen Street, the new address replaces the old one. The previous address would no longer be available in this table. Type 1 slowly changes dimension by overwriting previous values. + +Type 2: keep a history of address changes will have a table with the same customer and address columns, plus columns to show every version of the address and its validity, including the current address. Such columns would be address version, valid from, valid to, and is current. With these, we would be able to identify each address version, when it applies, and indicate whether it is the current address. This approach means that when a customer relocates, the store would keep the previous address row, record its end date, and mark it as no longer current. It would then add a new row containing the new address and its start date. The new row would be marked as current, with no end date until the address changes again. Thereby, allowing a customer to have several address records, but only one would be current. Therefore, Type 2 preserves previous addresses while recording new ones. ``` + *** ## Section 2: @@ -192,5 +205,17 @@ Consider, for example, concepts of labour, bias, LLM proliferation, moderating c ``` -Your thoughts... +Boykis's description of hidden work behind ImageNet has raised questions about whether we recognize the human effort or input in programming artificial intelligence (AI). Most people tend to celebrate AI's work, especially when they look at AI-generated images or viral AI videos for entertainment and to make some money, without appreciating the human labor that made it possible. For instance, consider how much effort goes into looking at the computer, from every graphic to every decision made about the image outputs. These human decisions on AI outputs may seem small, but together they help build a system that later receives praise for its intelligence. + +Considering fairness in the workplace as an ethical concern, researchers and AI implementers, including data modelers, deserve recognition for their workforce and meaningful contributions to the design of useful systems. Recognition from their organizations could take the form of time-reflective remuneration and, most importantly, the freedom to question unclear instructions and irregularities at work without being wrongly judged. Typically, this shows that fairness is more than just remuneration, especially when human effort is essential to decision-making. For example, in image classification, one person labels each image, whereas another decides which category it should be assigned to. + +In the article by Boykis, there were highlights on how human decisions are traced through ImageNet and earlier computer language resources. Thus, there is a sort of dependence on human decision and/or judgment as stated by Boykis: “every single piece of decision-making in a high-tech neural network initially rests on a human being” + +Consequently, deciding whether one appears trustworthy in a photograph does not establish honesty; this is the essence: accurate labeling is not always the right goal. This is because some decisions may be based on stereotypes rather than reliable evidence. Therefore, it is ethical to question both the appropriateness of the label and its availability where necessary. + +Furthermore, once these decisions are embedded in a system, human effort tends to diminish. And these are also what we face in the modern job sectors, where systems are automated to judge or assess job applicants. And tracing wrong judgments would be impossible except if those who created the training data, technology guides, and automated the system are also collectively involved in the selection process, rather than basing the outcome on an individual effort. This shows that most decisions made by a computer system are objective at times. + +Expanding the use of AI should prompt questions about the conditions under which human judgments are collected and whose perspectives those judgments represent, especially when it involves LLMs and content generation, to ensure the usefulness of content and the extent of harmful content generated or accessed. And ethically, the workers' rights should also be considered in the issues mentioned above. + +Generally, it is important to note that Boykis's article supports a thorough assessment and implementation of technologies rather than relying on finished technology. Thus, this encourages meaningful opportunities to challenge harmful decisions. Therefore, developers should also consider the affected communities when making decisions about capturing their lived experiences or needs that might otherwise be overlooked, because some stories cannot be told without the story's owner. However, consultation and better data cannot justify every application. We must also ask whether a particular judgment should be automated at all. A system can be technically impressive while still treating workers or the people it classifies unfairly. ``` diff --git a/02_activities/assignments/DC_Cohort/assignment1.sql b/02_activities/assignments/DC_Cohort/assignment1.sql index a80923591..1005c0a58 100644 --- a/02_activities/assignments/DC_Cohort/assignment1.sql +++ b/02_activities/assignments/DC_Cohort/assignment1.sql @@ -6,20 +6,16 @@ --SELECT /* 1. Write a query that returns everything in the customer table. */ --QUERY 1 - - - - +SELECT * FROM customer; --END QUERY /* 2. Write a query that displays all of the columns and 10 rows from the customer table, sorted by customer_last_name, then customer_first_ name. */ --QUERY 2 - - - - +SELECT * FROM customer +ORDER BY customer_last_name, customer_first_name +LIMIT 10; --END QUERY @@ -27,10 +23,9 @@ sorted by customer_last_name, then customer_first_ name. */ /* 1. Write a query that returns all customer purchases of product IDs 4 and 9. Limit to 25 rows of output. */ --QUERY 3 - - - - +SELECT * FROM customer_purchases +WHERE product_id IN (4, 9) +LIMIT 25; --END QUERY @@ -42,10 +37,11 @@ filtered by customer IDs between 8 and 10 (inclusive) using either: Limit to 25 rows of output. */ --QUERY 4 - - - - +SELECT *, + quantity * cost_per_quantity AS price +FROM customer_purchases +WHERE customer_id BETWEEN 8 AND 10 +LIMIT 25; --END QUERY @@ -55,10 +51,14 @@ Using the product table, write a query that outputs the product_id and product_n columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ --QUERY 5 - - - - +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product; --END QUERY @@ -67,9 +67,18 @@ add a column to the previous query called pepper_flag that outputs a 1 if the pr contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ --QUERY 6 - - - +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed, + CASE + WHEN LOWER(product_name) LIKE '%pepper%' THEN 1 + ELSE 0 + END AS pepper_flag +FROM product; --END QUERY @@ -78,10 +87,13 @@ contains the word “pepper” (regardless of capitalization), and otherwise out vendor_id field they both have in common, and sorts the result by market_date, then vendor_name. Limit to 24 rows of output. */ --QUERY 7 - - - - +SELECT + v.*, vba.* +FROM vendor AS v +INNER JOIN vendor_booth_assignments AS vba + ON v.vendor_id = vba.vendor_id +ORDER BY vba.market_date, v.vendor_name +LIMIT 24; --END QUERY @@ -92,10 +104,10 @@ Limit to 24 rows of output. */ /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ --QUERY 8 - - - - +SELECT vendor_id, + COUNT(*) AS booth_assignment_count +FROM vendor_booth_assignments +GROUP BY vendor_id; --END QUERY @@ -105,10 +117,19 @@ of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ --QUERY 9 - - - - +SELECT + c.customer_id, + c.customer_first_name, + c.customer_last_name, + SUM(cp.quantity * cp.cost_per_quantity) AS total_spent +FROM customer AS c +INNER JOIN customer_purchases AS cp + ON c.customer_id = cp.customer_id +GROUP BY c.customer_id, + c.customer_first_name, + c.customer_last_name +HAVING SUM(cp.quantity * cp.cost_per_quantity) > 2000 +ORDER BY c.customer_last_name, c.customer_first_name; --END QUERY @@ -121,13 +142,21 @@ When inserting the new vendor, you need to appropriately align the columns to be (there are five columns to be inserted, I've given you the details, but not the syntax) -> To insert the new row use VALUES, specifying the value you want for each column: -VALUES(col1,col2,col3,col4,col5) +VALUES(10,Thomass Superfood Store,Fresh Focused,Thomas,Rosenthal) */ --QUERY 10 - - - - +CREATE TABLE temp.new_vendor AS +SELECT * FROM vendor; + +INSERT INTO temp.new_vendor ( + vendor_id, + vendor_name, + vendor_type, + vendor_owner_first_name, + vendor_owner_last_name +) +VALUES (10, 'Thomass Superfood Store', 'Fresh Focused', + 'Thomas', 'Rosenthal'); --END QUERY @@ -138,10 +167,12 @@ HINT: you might need to search for strfrtime modifers sqlite on the web to know and year are! Limit to 25 rows of output. */ --QUERY 11 - - - - +SELECT + customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year +FROM customer_purchases +LIMIT 25; --END QUERY @@ -153,7 +184,15 @@ but remember, STRFTIME returns a STRING for your WHERE statement... AND be sure you remove the LIMIT from the previous query before aggregating!! */ --QUERY 12 - - - +SELECT + customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year, + SUM(quantity * cost_per_quantity) AS total_spent +FROM customer_purchases +WHERE STRFTIME('%m', market_date) = '04' + AND STRFTIME('%Y', market_date) = '2022' +GROUP BY customer_id, + STRFTIME('%m', market_date), + STRFTIME('%Y', market_date); --END QUERY diff --git a/02_activities/assignments/DC_Cohort/assignment2.sql b/02_activities/assignments/DC_Cohort/assignment2.sql index 4079c18ae..784170ae6 100644 --- a/02_activities/assignments/DC_Cohort/assignment2.sql +++ b/02_activities/assignments/DC_Cohort/assignment2.sql @@ -22,10 +22,11 @@ The `||` values concatenate the columns into strings. Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. All the other rows will remain the same. */ --QUERY 1 - - - - +SELECT + product_name || ', ' || + COALESCE(product_size, '') || ' (' || + COALESCE(product_qty_type, 'unit') || ')' AS product_details +FROM product; --END QUERY @@ -40,10 +41,15 @@ each new market date for each customer, or select only the unique market dates p HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). Filter the visits to dates before April 29, 2022. */ --QUERY 2 - - - - +SELECT + cp.*, + DENSE_RANK() OVER ( + PARTITION BY customer_id + ORDER BY market_date + ) AS visit_number +FROM customer_purchases AS cp +WHERE market_date < '2022-04-29' +ORDER BY customer_id, market_date; --END QUERY @@ -52,10 +58,29 @@ then write another query that uses this one as a subquery (or temp table) and fi only the customer’s most recent visit. HINT: Do not use the previous visit dates filter. */ --QUERY 3 - - - - +-- numbering the visits in reverse order. +SELECT + cp.*, + DENSE_RANK() OVER ( + PARTITION BY customer_id + ORDER BY market_date DESC + ) AS visit_number +FROM customer_purchases AS cp +ORDER BY customer_id, market_date DESC; + +-- most recent visit only. +SELECT * +FROM ( + SELECT + cp.*, + DENSE_RANK() OVER ( + PARTITION BY customer_id + ORDER BY market_date DESC + ) AS visit_number + FROM customer_purchases AS cp +) AS ranked_visits +WHERE visit_number = 1 +ORDER BY customer_id; --END QUERY @@ -65,10 +90,14 @@ customer_purchases table that indicates how many different times that customer h You can make this a running count by including an ORDER BY within the PARTITION BY if desired. Filter the visits to dates before April 29, 2022. */ --QUERY 4 - - - - +SELECT + cp.*, + COUNT(*) OVER ( + PARTITION BY customer_id, product_id + ) AS product_purchase_count +FROM customer_purchases AS cp +WHERE market_date < '2022-04-29' +ORDER BY customer_id, product_id, market_date; --END QUERY @@ -84,19 +113,33 @@ Remove any trailing or leading whitespaces. Don't just use a case statement for Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ --QUERY 5 - - - - +SELECT + product_name, + CASE + WHEN INSTR(product_name, '-') > 0 + THEN TRIM( + SUBSTR(product_name, INSTR(product_name, '-') + 1) + ) + ELSE NULL + END AS description +FROM product; --END QUERY /* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ --QUERY 6 - - - - +SELECT + product_name, + product_size, + CASE + WHEN INSTR(product_name, '-') > 0 + THEN TRIM( + SUBSTR(product_name, INSTR(product_name, '-') + 1) + ) + ELSE NULL + END AS description +FROM product +WHERE product_size REGEXP '[0-9]'; --END QUERY @@ -110,10 +153,40 @@ HINT: There are a possibly a few ways to do this query, but if you're struggling 3) Query the second temp table twice, once for the best day, once for the worst day, with a UNION binding them. */ --QUERY 7 - - - - +WITH daily_sales AS ( + SELECT + market_date, + SUM(quantity * cost_per_quantity) AS total_sales + FROM customer_purchases + GROUP BY market_date +), +ranked_sales AS ( + SELECT + market_date, + total_sales, + RANK() OVER ( + ORDER BY total_sales DESC + ) AS highest_rank, + RANK() OVER ( + ORDER BY total_sales ASC + ) AS lowest_rank + FROM daily_sales +) +SELECT + market_date, + total_sales, + 'Highest sales' AS sales_category +FROM ranked_sales +WHERE highest_rank = 1 + +UNION + +SELECT + market_date, + total_sales, + 'Lowest sales' AS sales_category +FROM ranked_sales +WHERE lowest_rank = 1; --END QUERY @@ -131,10 +204,42 @@ Think a bit about the row counts: how many distinct vendors, product names are t How many customers are there (y). Before your final group by you should have the product of those two queries (x*y). */ --QUERY 8 - - - - +WITH latest_vendor_products AS ( + SELECT + vi.vendor_id, + vi.product_id, + vi.original_price + FROM vendor_inventory AS vi + WHERE vi.market_date = ( + SELECT MAX(vi2.market_date) + FROM vendor_inventory AS vi2 + WHERE vi2.vendor_id = vi.vendor_id + AND vi2.product_id = vi.product_id + ) +), +potential_sales AS ( + SELECT + vp.vendor_id, + vp.product_id, + 5 * vp.original_price AS customer_sales + FROM latest_vendor_products AS vp + CROSS JOIN customer AS c +) +SELECT + v.vendor_name, + p.product_name, + SUM(ps.customer_sales) AS potential_revenue +FROM potential_sales AS ps +INNER JOIN vendor AS v + ON ps.vendor_id = v.vendor_id +INNER JOIN product AS p + ON ps.product_id = p.product_id +GROUP BY + v.vendor_id, + v.vendor_name, + p.product_id, + p.product_name +ORDER BY v.vendor_name, p.product_name; --END QUERY @@ -144,20 +249,37 @@ This table will contain only products where the `product_qty_type = 'unit'`. It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. Name the timestamp column `snapshot_timestamp`. */ --QUERY 9 - - - - +CREATE TABLE product_units AS +SELECT + p.*, + CURRENT_TIMESTAMP AS snapshot_timestamp +FROM product AS p +WHERE product_qty_type = 'unit'; --END QUERY /*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). This can be any product you desire (e.g. add another record for Apple Pie). */ --QUERY 10 - - - - +INSERT INTO product_units ( + product_id, + product_name, + product_size, + product_category_id, + product_qty_type, + snapshot_timestamp +) +SELECT + product_id, + product_name, + product_size, + product_category_id, + product_qty_type, + CURRENT_TIMESTAMP +FROM product +WHERE product_qty_type = 'unit' +ORDER BY product_id +LIMIT 1; --END QUERY @@ -166,10 +288,23 @@ This can be any product you desire (e.g. add another record for Apple Pie). */ HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ --QUERY 11 - - - - +DELETE FROM product_units +WHERE product_id = ( + SELECT MIN(product_id) + FROM product + WHERE product_qty_type = 'unit' +) +AND rowid <> ( + SELECT rowid + FROM product_units + WHERE product_id = ( + SELECT MIN(product_id) + FROM product + WHERE product_qty_type = 'unit' + ) + ORDER BY snapshot_timestamp DESC, rowid DESC + LIMIT 1 +); --END QUERY @@ -190,10 +325,24 @@ Finally, make sure you have a WHERE statement to update the right row, you'll need to use product_units.product_id to refer to the correct row within the product_units table. When you have all of these components, you can run the update statement. */ --QUERY 12 +ALTER TABLE product_units +ADD current_quantity INT; - - - +UPDATE product_units +SET current_quantity = COALESCE( + ( + SELECT SUM(vi.quantity) + FROM vendor_inventory AS vi + WHERE vi.product_id = product_units.product_id + AND vi.market_date = ( + SELECT MAX(vi2.market_date) + FROM vendor_inventory AS vi2 + WHERE vi2.product_id = product_units.product_id + ) + ), + 0 +) +WHERE product_qty_type = 'unit'; --END QUERY diff --git a/02_activities/assignments/DC_Cohort/images/Farmersmarket_conceptual_model.png b/02_activities/assignments/DC_Cohort/images/Farmersmarket_conceptual_model.png new file mode 100644 index 000000000..a4ea3e16b Binary files /dev/null and b/02_activities/assignments/DC_Cohort/images/Farmersmarket_conceptual_model.png differ diff --git a/02_activities/assignments/DC_Cohort/images/bookstore_logical_model_employee_shifts.drawio.png b/02_activities/assignments/DC_Cohort/images/bookstore_logical_model_employee_shifts.drawio.png new file mode 100644 index 000000000..7a102eb7b Binary files /dev/null and b/02_activities/assignments/DC_Cohort/images/bookstore_logical_model_employee_shifts.drawio.png differ diff --git a/02_activities/assignments/DC_Cohort/images/small_bookstore_logical_model.drawio.png b/02_activities/assignments/DC_Cohort/images/small_bookstore_logical_model.drawio.png new file mode 100644 index 000000000..82ca81b2f Binary files /dev/null and b/02_activities/assignments/DC_Cohort/images/small_bookstore_logical_model.drawio.png differ diff --git a/farmersmarket.db b/farmersmarket.db new file mode 100644 index 000000000..cdce82204 Binary files /dev/null and b/farmersmarket.db differ diff --git a/farmersmarket.sqbpro b/farmersmarket.sqbpro new file mode 100644 index 000000000..8e0392bc7 --- /dev/null +++ b/farmersmarket.sqbpro @@ -0,0 +1,104 @@ +SELECT * FROM customer; + +SELECT * FROM customer + ORDER BY customer_last_name, customer_first_name + LIMIT 10; + +SELECT * FROM customer_purchases + WHERE product_id IN (4, 9) + LIMIT 25; + +SELECT *, + quantity * cost_per_quantity AS price +FROM customer_purchases +WHERE customer_id BETWEEN 8 AND 10 +LIMIT 25; + + +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product; + + +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed, + CASE + WHEN LOWER(product_name) LIKE '%pepper%' THEN 1 + ELSE 0 + END AS pepper +FROM product; + + +SELECT + v.*, vba.* +FROM vendor AS v +INNER JOIN vendor_booth_assignments AS vba + ON v.vendor_id = vba.vendor_id +ORDER BY vba.market_date, v.vendor_name +LIMIT 24; + + +SELECT vendor_id, + COUNT(*) AS booth_assignment_count +FROM vendor_booth_assignments +GROUP BY vendor_id; + + +SELECT + c.customer_id, + c.customer_first_name, + c.customer_last_name, + SUM(cp.quantity * cp.cost_per_quantity) AS total_spent +FROM customer AS c +INNER JOIN customer_purchases AS cp + ON c.customer_id = cp.customer_id +GROUP BY c.customer_id, + c.customer_first_name, + c.customer_last_name +HAVING SUM(cp.quantity * cp.cost_per_quantity) > 2000 +ORDER BY c.customer_last_name, c.customer_first_name; + + +CREATE TABLE temp.new_vendor AS +SELECT * FROM vendor; + +INSERT INTO temp.new_vendor ( + vendor_id, + vendor_name, + vendor_type, + vendor_owner_first_name, + vendor_owner_last_name +) +VALUES (10, 'Thomass Superfood Store', 'Fresh Focused', + 'Thomas', 'Rosenthal'); + + +SELECT + customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year +FROM customer_purchases +LIMIT 25; + + +SELECT + customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year, + SUM(quantity * cost_per_quantity) AS total_spent +FROM customer_purchases +WHERE STRFTIME('%m', market_date) = '04' + AND STRFTIME('%Y', market_date) = '2022' +GROUP BY customer_id, + STRFTIME('%m', market_date), + STRFTIME('%Y', market_date);