Merge pull request #43 from ruralinnovation/dev/AHR-103-share

AHR-103 Share latest diagrams in team wiki

MDA-Data-Infrastructure-Overview.qmd

@@ -7,15 +7,27 @@ execute:
   warning: false
 ---
 
-<div style="background-color: #fff"><a href="" target="_blank" style="background-color: #fff">
-![](img/MDA%20Data%20Infrastructure.svg)
+## Current
+
+<div style="background-color: #fff">
+<a href="" target="_blank" style="background-color: #fff">
+![](img/MDA%20Data%20Flow%20-%20Current.drawio.svg)
 <!--
-<img src="MDA%20Data%20Infrastructure.svg" alt="mermaid-diagram" style="background-color: #fff" width="100%" />
+<img src="img/MDA%20Data%20Flow%20-%20Current.drawio.svg" alt="mermaid-diagram" style="background-color: #fff" width="100%" />
 -->
 </a>
 </div>
 <br/>
 
+
+## Future
+
+<div style="background-color: #fff">
+<a href="" target="_blank" style="background-color: #fff">
+![](img/MDA%20Data%20Flow%20-%20Future.drawio.svg)
 <!--
-![](MDA%20Data%20Flow.svg
--->
+<img src="img/MDA%20Data%20Flow%20-%20Future.drawio.svg" alt="mermaid-diagram" style="background-color: #fff" width="100%" />
+-->
+</a>
+</div>
+<br/>

Metadata.md

@@ -1,8 +1,13 @@
+# How ? 
+
+
+
+
 ## What is Metadata?
 
 The MDA tracks a variety of metadata on high-traffic tables. Metadata is information _about_ the data that we are sourcing and/or creating, which may include:
   * Where the data came from (source)
-  * When the data was acquired
+  * When the data was acquired -> 
   * What the table names mean
   * What the field names mean
   * Any information about how the data was created/generated/derived
@@ -85,3 +90,16 @@ present in 3 tables and save a 3 csv for them with select row
 -  update our tools in production: BEAD / CH 
 - cori.utils check
 - side not unsure if dplyr::add_rows should be added here
+
+First steps: 
+
+- we should focus first on:
+
+  * tables in DB
+
+  * bucket in s3
+
+- setup automation for that (write db)
+
+- see what we do with pull data that is writing to temp/disk  
+

_quarto.yml

@@ -21,23 +21,24 @@ website:
           - CORI-Ontology.qmd
       - section: "Infrastructure"
         contents:
+          - MDA-Data-Infrastructure-Overview.qmd
           - Ansible.md
           - CORI-Data-API.md
-          #          - section: "CORI Data API"
-          #            contents:
-          #              - CORI-Data-API.md
-          ##              - Adding-New-Endpoints-&-Services-to-the-Python-RESTApi.md
-          ##              - Apollo-Studio-Setup.md
-          ##              - Configuring-our-AWS-CLI-(CDK-and-SAM)-Credentials.md
-          ##              - Example-React-Map-Container-with-REACT-MAP-GL-Package.md
-          ##              - Migrating-from-CDK-V1-=--V2.md
-          ##              - Postman-Workspace-Setup-and-Configuration.md
-          ##              - R-and-Python-API-Usage.md
-          ##              - React,-AWS-Amplify-and-API-Usage.md
-          ##              - SAML-User-Pool-IdP-Authentication-Flow.md
-          ##              - Session-2-‐-2023‐10‐23.md
-          ##              - Support-Sessions.md
-          ##              - Working-with-the-GraphQL-Schemas.md
+        #  - section: "CORI Data API"
+        #    contents:
+        #      - CORI-Data-API.md
+        ##      - Adding-New-Endpoints-&-Services-to-the-Python-RESTApi.md
+        ##      - Apollo-Studio-Setup.md
+        ##      - Configuring-our-AWS-CLI-(CDK-and-SAM)-Credentials.md
+        ##      - Example-React-Map-Container-with-REACT-MAP-GL-Package.md
+        ##      - Migrating-from-CDK-V1-=--V2.md
+        ##      - Postman-Workspace-Setup-and-Configuration.md
+        ##      - R-and-Python-API-Usage.md
+        ##      - React,-AWS-Amplify-and-API-Usage.md
+        ##      - SAML-User-Pool-IdP-Authentication-Flow.md
+        ##      - Session-2-‐-2023‐10‐23.md
+        ##      - Support-Sessions.md
+        ##      - Working-with-the-GraphQL-Schemas.md
           - Data-Security.qmd
           - duckDB.qmd
           - PostgreSQL-RDS-Managment.md