AI Data Prep


Capturing important metadata to optimize AI performance.

Project Overview

In order for AI to properly utilize data sources, they must contain important meta data. This meta data includes things like column descriptions, and common words that users might use when referring to specific data. With so many of our users wanting to use AI, data prep became an increasingly important first step to ensuring they had a good experience with AI Chat and other products. This project involved merging an existing tab called ‘schema’ with the new features we needed for ‘AI Ready’ datasets.

My Role

I met with development leads and project manager to discuss the goals and objectives of the project. I then re-designed a data tab to provide insight around AI data prep for individual data sets. Using the existing schema table design, I incorporated the ability to add important AI meta data. I also designed data prep cards to indicate the meta data added and the overall ‘readiness’ of the dataset.

Year:

2025

Timeline:

approx. 1 week

Readiness indicators

While AI needs important meta data to correctly interpret a dataset, it’s possible to overload the AI with too much, unhelpful information. Adding meta data also required a fair amount of time from the user, so it was important to indicate how much information was really needed.


  • Summary cards on both the dataset detail page on AI ready tab show a quick status of different AI ready factors.

  • Colored bar charts on the first two cards provide a quick visual indicator of a dataset’s ‘readiness’.

  • ‘AI enabled’ columns show if the AI has enough meta data to go off of, or if there’s too much.

  • Characters in context also provide a signal to the user on whether or not there has been the appropriate amount of meta data provided to the AI.

Prepping a dataset

Once on the AI Ready tab, users would be presented with the columns of their dataset and the information that had or had not been added. Column context, context character count, synonyms and ‘beast modes’ all impact how the AI can read and understands the dataset.


  • Users can edit AI meta data in the same place they find the column configuration settings they’re already used to.

  • Less disorienting for the user to have different default chat modes, than present users with a new agent on every page.

  • Users can add context for the overall dataset as well as individual dataset columns.

  • Users can filter by AI enabled dataset columns in order to better manage large datasets.

  • Table editing is presented in a familiar way, very similar to the way Excel edits spreadsheets.

AI Ready datasets

Once a prompt is entered, the AI displays it’s thought process with sources, tools, and expandable code. This is an animated process that runs before the AI populates its answer. Displaying the thinking process not only helps users see into the back box of AI, but also provides an opportunity to stop the AI if the user sees it going down the wrong road. 



  • Once meta data has been added, the table cell updates with the new status.

  • Synonyms display the total number of synonyms added, while AI context displays the added context on hover.

  • Summary cards at the top update to reflect changes. The banner at the top also provides a success message once the dataset has enough meta data.

  • Any sources the AI used in it’s thinking process are kept at the top of the response for quick reference.