New view mobile search clients daily

[alekhyamoz]

This is related to a query change we are making wrt the mobile search tables. Previously the metrics were pulled from metrics ping but now we are in the process of switching the query to pull from baseline ping. The cut off date will be 2024.08.01 i.e. all the data on and before 2024.07.31 pulls from metrics and after 2024.07.31 pulls from baseline. The validation is in progress by the DS team.

This PR creates the view mobile_search_clients_daily that reads from mobile_search_clients_daily_historical_pre202408 until 2024-07-31 and mobile_search_clients_daily_v2 from 2024-08-01
A follow-up PR to switch all existing tables to this new view
and also point mobile_search_clients_engines_sources_daily to new table mobile_search_clients_daily_v2

Related Tickets & Documents

• DENG-XXXX
• DSRE-XXXX

Reviewer, please follow this checklist

┆Issue is synchronized with this Jira Task

[dataops-ci-bot]

Integration report for "Merge branch 'main' into new_view_mobile_search_clients_daily"

sql.diff

Click to expand!

Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search: mobile_search_clients_daily
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/metadata.yaml	2024-10-18 19:34:08.000000000 +0000
@@ -0,0 +1,18 @@
+friendly_name: Mobile Search Clients Daily
+description: |
+  A daily aggregate of baseline and metrics pings that have
+  searches from each (client_id, engine, source) combination,
+  partitioned by day.
+owners:
+- [email protected]
+labels:
+  owner1: akommasani
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_historical_pre202408
+  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v2
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_daily/view.sql	2024-10-18 19:31:54.000000000 +0000
@@ -0,0 +1,96 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
+AS
+SELECT
+  submission_date,
+  client_id,
+  engine,
+  source,
+  app_name,
+  search_count,
+  organic,
+  tagged_sap,
+  tagged_follow_on,
+  ad_click,
+  search_with_ads,
+  unknown,
+  country,
+  locale,
+  app_version,
+  channel,
+  os,
+  os_version,
+  default_search_engine,
+  default_search_engine_submission_url,
+  distribution_id,
+  profile_creation_date,
+  profile_age_in_days,
+  sample_id,
+  experiments,
+  total_uri_count,
+  ad_click_organic,
+  search_with_ads_organic,
+  os_version_major,
+  os_version_minor,
+  `moz-fx-data-shared-prod`.udf.normalize_search_engine(engine) AS normalized_engine,
+  `mozfun.mobile_search.normalize_app_name`(
+    app_name,
+    os
+  ).normalized_app_name AS normalized_app_name,
+  `mozfun.norm.browser_version_info`(app_version) AS browser_version_info,
+  search_count AS sap,
+  `mozfun.mobile_search.normalize_app_name`(
+    app_name,
+    os
+  ).normalized_app_name AS normalized_app_name_os
+FROM
+  `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_historical_pre202408`
+WHERE
+  submission_date <= '2024-07-31'
+UNION ALL
+SELECT
+  submission_date,
+  client_id,
+  engine,
+  source,
+  app_name,
+  search_count,
+  organic,
+  tagged_sap,
+  tagged_follow_on,
+  ad_click,
+  search_with_ads,
+  unknown,
+  country,
+  locale,
+  app_version,
+  channel,
+  os,
+  os_version,
+  default_search_engine,
+  default_search_engine_submission_url,
+  distribution_id,
+  profile_creation_date,
+  profile_age_in_days,
+  sample_id,
+  experiments,
+  total_uri_count,
+  ad_click_organic,
+  search_with_ads_organic,
+  os_version_major,
+  os_version_minor,
+  `moz-fx-data-shared-prod`.udf.normalize_search_engine(engine) AS normalized_engine,
+  `mozfun.mobile_search.normalize_app_name`(
+    app_name,
+    os
+  ).normalized_app_name AS normalized_app_name,
+  `mozfun.norm.browser_version_info`(app_version) AS browser_version_info,
+  search_count AS sap,
+  `mozfun.mobile_search.normalize_app_name`(
+    app_name,
+    os
+  ).normalized_app_name AS normalized_app_name_os
+FROM
+  `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v2`
+WHERE
+  submission_date > '2024-07-31'

Link to full diff