Skip to content
Draft
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
14ba547
docs: simplify big number block introduction
Haleshot Aug 28, 2026
d4c568c
docs: clean up bigquery-oauth prose
Haleshot Aug 28, 2026
72fcd81
docs: clean up billing-alerts-and-limits prose
Haleshot Aug 28, 2026
b105d8f
docs: clean up chart-blocks prose
Haleshot Aug 28, 2026
7e4db15
docs: clean up conda-environment prose
Haleshot Aug 28, 2026
ce950b2
docs: clean up converting-notebooks prose
Haleshot Aug 28, 2026
67d9f04
docs: clean up custom-environment prose
Haleshot Aug 28, 2026
bb21990
docs: replace en dash in data app guide
Haleshot Aug 28, 2026
9d654fb
docs: rewrite data catalog guide
Haleshot Aug 28, 2026
4289585
docs: remove decorative emoji from dbt
Haleshot Aug 28, 2026
6777d72
docs: clean up deepnote-agent prose
Haleshot Aug 28, 2026
0927316
docs: clean up deepnote-ai prose
Haleshot Aug 28, 2026
5e44c93
docs: clean up deepnote-file-sync prose
Haleshot Aug 28, 2026
d668ebc
docs: clean up deepnote-first-steps prose
Haleshot Aug 28, 2026
2109607
docs: clean up deepnote-mcp prose
Haleshot Aug 28, 2026
8b385ff
docs: remove decorative emoji from deploying-machine-learning-models
Haleshot Aug 28, 2026
d16a70d
docs: remove decorative emoji from deprecating-personal-workspaces
Haleshot Aug 28, 2026
dd710e3
docs: replace double hyphen in Docker Hub guide
Haleshot Aug 28, 2026
9c22101
docs: clean up edu-overview prose
Haleshot Aug 28, 2026
19943d8
docs: rewrite ETL and ELT pipeline guide
Haleshot Aug 28, 2026
b2e93a9
docs: clean up export-pdf prose
Haleshot Aug 28, 2026
ef1f328
docs: tighten getting started copy
Haleshot Aug 28, 2026
d211e00
docs: clean up git-export prose
Haleshot Aug 28, 2026
7418c3a
docs: remove decorative emoji from google-drive
Haleshot Aug 28, 2026
4a605d4
docs: clean up history prose
Haleshot Aug 28, 2026
6fb46ad
docs: tighten docs landing page
Haleshot Aug 28, 2026
4c7b860
docs: clean up individual-assignments prose
Haleshot Aug 28, 2026
d717120
docs: clean up input-blocks prose
Haleshot Aug 28, 2026
6c59f54
docs: remove decorative emoji from integrations
Haleshot Aug 28, 2026
13a8f94
docs: clean up ipywidgets-in-deepnote prose
Haleshot Aug 28, 2026
1b0438d
docs: remove emoji from shortcut headings
Haleshot Aug 28, 2026
bd99308
docs: label URL Encoder link
Haleshot Aug 28, 2026
fe4d340
docs: clean up lectures prose
Haleshot Aug 28, 2026
c827a4a
docs: remove decorative emoji from local setup
Haleshot Aug 28, 2026
f570616
docs: clean up long-running-jobs prose
Haleshot Aug 28, 2026
19ee4da
docs: remove emoji from conversion lists
Haleshot Aug 28, 2026
12eefa8
docs: clean up modules prose
Haleshot Aug 28, 2026
6270c27
docs: remove emoji from notebook headings
Haleshot Aug 28, 2026
521d453
docs: clean up pricing prose
Haleshot Aug 28, 2026
9400ae7
docs: clean up projects prose
Haleshot Aug 28, 2026
d23935d
docs: remove decorative emoji from real-time-collaboration
Haleshot Aug 28, 2026
362c120
docs: clean up run-snapshots prose
Haleshot Aug 28, 2026
ba15545
docs: clean up scheduling prose
Haleshot Aug 28, 2026
ba13ea9
docs: clean up securing-connections prose
Haleshot Aug 28, 2026
3221170
docs: rewrite security overview
Haleshot Aug 28, 2026
e828466
docs: replace en dash in sharing guide
Haleshot Aug 28, 2026
63618a9
docs: clean up snowflake-oauth prose
Haleshot Aug 28, 2026
e5fdfff
docs: clean up snowflake-with-azure-ad prose
Haleshot Aug 28, 2026
ed0aaeb
docs: clean up snowflake-with-okta prose
Haleshot Aug 28, 2026
674ce3b
docs: clean up snowpark prose
Haleshot Aug 28, 2026
35076b8
docs: replace en dash in support guide
Haleshot Aug 28, 2026
a1eef8f
docs: tighten teacher guide
Haleshot Aug 28, 2026
ca29df1
docs: replace en dash in permissions guide
Haleshot Aug 28, 2026
5ac3a64
docs: clean up telegram prose
Haleshot Aug 28, 2026
7b3b8a9
docs: clean up text-editing prose
Haleshot Aug 28, 2026
26fa90f
docs: remove decorative emoji from training-machine-learning-models
Haleshot Aug 28, 2026
6ad745f
docs: clean up workspaces prose
Haleshot Aug 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs: rewrite data catalog guide
Signed-off-by: Srihari Thyagarajan <hari.leo03@gmail.com>
  • Loading branch information
Haleshot committed Aug 28, 2026
commit 9d654fb6e0f642880e31b9fa175b96c81f3ecab8
86 changes: 16 additions & 70 deletions docs/data-catalogs.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,90 +4,36 @@ noIndex: false
noContent: false
---

Data cataloging is a game-changer in modern data management. At the forefront of this revolution is Deepnote, an AI-powered data workspace. Let’s dive into data cataloging, the unique capabilities of Deepnote, and how it empowers data scientists, analysts, and engineers to achieve greater insights and efficiency.
A data catalog records an organization's data assets and the metadata that describes them. It helps users find a dataset, understand its source and schema, assess its quality, and check who can access it.

**What is a data catalog?** A data catalog is a comprehensive inventory of an organization's data assets. It uses metadata aka data about data, to provide detailed descriptions, helping users quickly locate, evaluate, and understand the data they need. This inventory includes structured data like tables and charts, unstructured data such as documents and images, and machine learning models and visualizations. Key components of a data catalog:
A catalog can include tables, charts, documents, images, machine learning models, and visualizations. Search, tags, filters, lineage, quality metrics, and access controls make those assets easier to find and evaluate.

- **Metadata management**: collects and organizes metadata to describe data assets.
- **Search functionality**: facilitates easy discovery of data through keyword search, tagging, and filters.
- **Data evaluation**: provides context and quality metrics to assess data suitability for specific uses.
- **Access control**: ensures secure access to data, maintaining compliance with regulations.
## Build a data catalog in Deepnote

## Why data cataloging is crucial
Use Markdown blocks to document each dataset's source, creation date, schema, and purpose. Profile the data to find missing values, outliers, or inconsistencies, and add summary statistics or visualizations when they help readers understand the dataset.

Implementing a robust data catalog offers numerous benefits:
Folders and tags can organize related datasets. Deepnote's version history records changes to the notebook, so your team can review updates to the catalog over time.

- **Improved efficiency**: streamlines data discovery and reduces time spent searching for data.
- **Enhanced data context**: provides comprehensive metadata that helps users understand data relevance and quality.
- **Reduced errors**: ensures data accuracy by maintaining detailed data lineage and quality information.
- **Better decision-making**: facilitates quicker and more informed decisions by providing easy access to high-quality data.

## Deepnote's approach to data cataloging

Deepnote takes data cataloging to the next level by offering a seamless, collaborative, and intuitive platform specifically designed for data professionals.

**Documenting and describing datasets**: use Markdown blocks to document each dataset, including metadata such as source, creation date, and data schema. Detailed descriptions maintain clarity and context.

**Data profiling and cleaning**: identify and correct missing values, outliers, and inconsistencies through data profiling. This ensures the quality and integrity of your datasets.

**Incorporating visualizations and summary statistics**: add visualizations using libraries like Seaborn to provide a quick overview of the data. This helps in understanding data patterns and making informed decisions.

**Collaborating with team members**: share your Deepnote project and set permissions for team members to ensure effective teamwork and data integrity, or create a [Deepnote app](https://deepnote.com/docs/data-apps) that you embedded in a document for your coworkers.

**Organizing and managing datasets**: use folders and tags to organize your project. This ensures a streamlined workflow and easy retrieval of datasets.

**Maintaining an up-to-date data catalog**: regularly updating documentation and metadata. Use version control to track changes and set periodic reviews to ensure data accuracy and relevance.

Deepnote offers several unique ways that make it an ideal platform for data cataloging:

- **Seamless integration**: integrates with various data sources, allowing easy import and management of datasets.
- **Collaborative environment**: enables real-time teamwork with robust sharing and permission-setting features.
- **Interactive visualizations**: provides tools to create visualizations and summary statistics for quick data insights.
- **Enhanced documentation**: utilizes Markdown blocks for comprehensive documentation, ensuring clarity and context.
- **Interactive data applications**: allows the creation of interactive data applications within Deepnote, streamlining the cataloging process.

By integrating Deepnote into your data management workflow, you can significantly enhance your ability to catalog, manage, and analyze datasets. This leads to better decision-making, increased efficiency, and a deeper understanding of your data assets.

- **Empowers users**: Deepnote's intuitive interface and powerful features empower data professionals to take control of their data cataloging processes.
- **Promotes collaboration**: encourages teamwork and knowledge sharing, fostering a collaborative data culture within organizations.
- **Drives innovation**: by simplifying complex processes, Deepnote allows data professionals to focus on innovation and derive actionable insights from their data.
Share the project with teammates and set view or edit permissions for each collaborator. You can also publish the catalog as a [Deepnote app](https://deepnote.com/docs/data-apps) and embed it in another document.

## Example of data cataloging in Deepnote

We prepared an [example notebook](https://deepnote.com/workspace/Deepnote-Templates-71742312-24f2-4c10-9bf7-786d17280b92/project/Data-catalog-34f035b4-bb25-410b-ac79-5c60c6ccc5cb/notebook/%F0%9F%93%9D%20Introduction%20-f00a6fdd779148238e351b26646bcfe7) with [Atlan API](https://atlan.com/) that you can duplicate, play around, or use for your data set. There are multiple levels that you can explore depends on your seniority and use case.
The [example data catalog notebook](https://deepnote.com/workspace/Deepnote-Templates-71742312-24f2-4c10-9bf7-786d17280b92/project/Data-catalog-34f035b4-bb25-410b-ac79-5c60c6ccc5cb/notebook/%F0%9F%93%9D%20Introduction%20-f00a6fdd779148238e351b26646bcfe7) uses the [Atlan API](https://atlan.com/). Duplicate it to explore the setup or adapt it to your datasets.

## Collaborating and sharing your data catalog

**Share your notebook**

- Click on the “Share” button in the top-right corner of the notebook.
- Set permissions for your collaborators (e.g., view, edit).
- Share the link with your team.
- Create a [data app](https://deepnote.com/docs/data-apps) from your data catalog
1. Click **Share** in the top-right corner of the notebook.
2. Set view or edit permissions for each collaborator.
3. Share the link with your team, or create a [data app](https://deepnote.com/docs/data-apps) from the catalog.

**Collaborate in real-time**

- Collaborate with your team by allowing them to add, edit, or comment on the notebook.
- Use comments to discuss dataset details or flag issues that need attention.
Collaborators can add, edit, or comment on the notebook. Use comments to discuss dataset details or flag issues.

## Maintaining and updating your data catalog

**Regular updates**: set a schedule for reviewing and updating your data catalog. This ensures that all information is current and accurate.

Example: Every month, review the datasets and update any changes in schema, data sources, or descriptions.

**Version control:** utilize Deepnote’s version control features to track changes in your notebook.

This helps maintain a history of updates and allows you to revert to previous versions if needed.

In the rapidly evolving landscape of data management, having a robust data catalog is no longer optional—it's essential. Deepnote not only makes data cataloging more accessible but also enhances it with features tailored to the needs of data professionals. By leveraging Deepnote's capabilities, you can transform your data cataloging process, driving better decision-making and unlocking the full potential of your data.

Ready to streamline your data cataloging process? Sign up for Deepnote today and experience the benefits firsthand. Empower your data teams, enhance collaboration, and drive innovation with Deepnote—the leading platform for data cataloging.

For those interested in further exploring the capabilities of Deepnote and how it can revolutionize your data management processes, here are some additional resources:
Set a review schedule for the catalog. During each review, record changes to schemas, data sources, and descriptions. Use version history to inspect past updates or restore an earlier version.

- [**Getting started with Deepnote**](https://deepnote.com/docs): Learn the basics of setting up and using Deepnote for your data projects.
- [**Deepnote integrations**](https://deepnote.com/integrations): Discover how to connect Deepnote with your existing data sources and tools.
- [**Case studies**](https://deepnote.com/customers): See how other organizations are using Deepnote to enhance their data practices and drive success.
## Related resources

Feel free to reach out to our support team if you have any questions or need assistance getting started with Deepnote. Happy data cataloging!
- [Getting started with Deepnote](https://deepnote.com/docs)
- [Deepnote integrations](https://deepnote.com/integrations)
- [Customer stories](https://deepnote.com/customers)