⭐0.1.4

Enhanced treemap with new features, lineage graph integration and UI updates.

What's Changed

Frontend Service

New Features and Enhancements:

• Treemap Visualization Updates:

  • Improved treemap with the display of types, children count on tiles, and modification to the copied node's width.

  • Added base dataset name and similar dataset details at the top of the treemap.

  • Introduced legend colors at the top of the treemap for better data interpretation.

  • Updated treemap to version 0.2.5 for enhanced features and functionality.

• Lineage Graph Integration:

  • Implemented setup for showing the lineage graph on the side panel, enhancing data navigation and visualization.

  • Updated functionality to show only the highlighted path and nodes when clicking on a particular node in the lineage graph.

UI/UX Improvements:

• Inventory and Capacity Table Enhancements:

  • Made minor updates to capacity table columns on the impact page and inventory card tooltip, improving data display and user interaction.

  • Updated the tooltip to be visible only on mouse hover over the icon.

• Color Scheme and Accessibility:

  • Revised treemap color scheme to be colorblind safe, ensuring accessibility for all users.

Gateway Service

New Features and Enhancements:

• Treemap Schema Update: Added "count_children" integer value to treemap schemas, enhancing the frontend's capability to display hierarchical data more effectively.

• Snowflake Proxy Enhancements:

  • Introduced new methods in the Snowflake proxy specifically for duplicate datasets treemap, improving data handling and visualization capabilities.

  • Fixed handling of null children in the Snowflake BI proxy, ensuring better data integrity and error handling.

UI/UX Improvements:

• Capacity Table Update on Impact Page: Made a minor update to the capacity table columns on the impact page, likely for better data representation and user understanding.

Data Model and API Updates:

• RecommendationPreview Model Enhancements:

  • Added workspace_id and workspace_name to the RecommendationPreview Pydantic model, enriching the data model with more contextually relevant information.

  • Implemented workspace_id and workspace_name in the /preview endpoint, aligning the API's output with the updated data model.

Diagnostics Service

New Features and Enhancements:

• Treemap Visualization Updates:

  • Added "removed" objects to the treemap, enhancing data representation.

  • Updated the Impact Page to show all months of capacity impact in a single table.

  • Simplified calculation for the count of duplicate datasets for PowerBI.

• Anonymization and Data Privacy:

  • Implemented anonymization of metric descriptions and other data for demo connectors.

  • Added more anonymizer functions for creating fake data, supporting nested JSON structures.

UI/UX Improvements:

• Tableau API Hardening: Strengthened the Tableau API extraction process for better data integrity and reliability.

• Minor Updates to Metric Descriptions: Adjusted metric descriptions for PowerBI datasource and Tableau dataset to remove PII and ensure more controlled anonymization.

Operational and Infrastructure Changes:

• Dockerfile and Deployment Updates:

  • Multiple updates to Dockerfile for Airflow images to ensure the vault folder is writable by the Airflow user.

  • Adjusted demo data to meet gateway Pydantic model requirements for issue preview data.

• Development Integrations and Merge Resolutions: Multiple merges from the development branch, integrating features and resolving conflicts.

• SQL and Model Refinements:

  • Various SQL adjustments (referred to as 'fluffing') and model updates, including refresh history model remake.

  • Fixed a bug related to unreasonably high refresh counts.