Prompting AI for pipelines.
Breaking barriers to simplify data transformation with AI-driven UX.
TL:DR
To stay ahead in the AI-driven SaaS landscape, Matillion set out to create an AI-powered proof of concept for the Snowflake Summit, aiming to enhance user data productivity.
Beginning with a hackathon, our team rapidly turned ideas into an innovative demo, showcasing how AI could transform workflows within the Data Productivity Cloud.
Overseeing the end-to-end process in collaboration with:
2x Designers (Senior & Junior)
Product manager
Product owner
Front & back-end engineering
Leading to significant outcomes
25% reduction in pipeline creation time.
17% new feature adoption rate.
42% task completion rate.
Matillion
April 2024
4 months
Large
Design manager
Managing designers
Stakeholder management
Vision & strategy
Cross-functional alignment
Decision making
Project timelines & tracking
Team growth & development
Created for
Date
Duration
Effort
Role
Deliverables
Background
Matillion is a SaaS B-2-B cloud data transformation tool.
Data productivity cloud, the principal offering from Matillion allows users to connect various data sources, perform transformations and then deliver that data to their
warehouse of choice. Commonly referred to as ETL (extract, transform & load.)
The goal is to provide this service as a low/no code
solution, reducing the needs and cost of data engineers within an organisation by removing the technical complexity of hand coding.
Brief
With AI rapidly transforming the SaaS landscape, Matillion saw a crucial opportunity “Harness AI to empower users in their data workflows.” This journey began with a team-wide hackathon, where we explored AI’s potential across our platform. The outcome was clear. “AI could make data productivity faster, smarter, and more intuitive.”
The Snowflake Summit, just four months away, became our ambitious target to showcase this vision. Our mission? Design an AI-powered proof of concept that would give users a glimpse of the future in data productivity, all while driving Matillion’s commitment to innovation.
Engineers were in full swing building the core tech, and our team focused on shaping it into a demo ready to make waves at the Summit.
Stakeholder interviews
In collaboration with designers & project manager.
For the project kickoff, we faced numerous stakeholders with varied opinions and ideas.
Mapping stakeholders on an axis of advocate vs. opponent and low vs. high engagement provided a clear strategy for where to invest efforts for buy-in and who would offer consistent support.
This approach enabled us to leverage highly engaged advocates as key promoters of the product.
Competitor analysis
In collaboration with the designers.
To make Copilot stand out in the AI-driven SaaS landscape, we conducted competitor research to explore how others used AI within data transformation tool and comparable canvas/workflow tools.
This revealed a common trend across tools: leveraging AI to simplify workflows and remove barriers.
Inspired by this, we shaped our solution to empower users in building pipelines with our canvas tool, aligning with Matillion’s mission to make data transformation seamless.
In collaboration with designers & project manager.
With limited time, we joined sales and customer success calls to quickly gather insights into users’ pipeline-building needs and pain points. Users shared frustrations around repetitive tasks “Copilot automating even a portion would save hours” and the complexity of transformations, with one saying, “A tool that could guide me through complex steps would be a game-changer.”
Others highlighted the need for simplicity for non-coders and control over critical data “Having a ‘human in the loop’ lets me ensure accuracy.” These user insights gave us a clear direction to follow and valuable findings to share with the business, helping secure buy-in for Copilot’s AI features.
User interviews
“Building and configuring data pipelines is complex and time-consuming for both new and returning users, leading to frustration and inefficiency. Copilot needs to provide intuitive, AI-driven guidance to simplify this process, reducing the learning curve and empowering users to complete tasks confidently.”
Playback & alignment
In collaboration with designers & project manager.
To align challenging stakeholders, I tailored discussions to their interests, focusing on the current user experience and how an enhanced Copilot could drive adoption and increase pipeline creation, a key business objective.
The problem we aimed to solve was the complexity of building and configuring a pipeline whether for new users onboarding to the platform or experienced users looking to streamline their workflow. Drawing on user interviews, stories, and empathy maps, I showcased real pain points and highlighted how Copilot could benefit both users and business metrics, securing alignment around a shared, user-centred vision.
Success metrics
Reduce pipeline creation time by 30%.
This aligns with Copilot’s core objective of minimising manual effort and saving users time, ultimately reducing costs and workload for data teams.
Adoption rate of 20% in first 3 months.
Adoption is crucial for demonstrating Copilot’s value, and tracking this metric will guide future improvements and user education efforts to maximise reach.
60% task completion rate.
This metric helps ensure that Copilot’s guidance is effective in supporting users, even in complex scenarios, thereby boosting confidence in the tool and increasing task success rates.
20% reduction in support content usage.
This goal highlights Copilot’s role in simplifying pipeline creation and a self-sufficient tool, equipping users with solutions to common issues.s and workload for data teams.
Reduction in onboarding time by 20%.
This goal supports our vision of making pipeline-building more approachable, ultimately expanding Copilot’s appeal to a broader range of users and increasing satisfaction from the start.
Ideation
In collaboration with designers.
We organised an ideation workshop with challenging stakeholders,
where I mentored a designer to lead a sketching session to build their facilitation skills. To set the context, we provided a pre-read and used
an icebreaker sketching exercise to help hesitant participants feel comfortable visualising ideas.
With opinionated stakeholders, “How Might We” questions kept the
focus on the right problem, guiding constructive idea-sharing. This approach fostered collaboration, gave stakeholders a sense of
ownership, and provided a valuable growth experience for the
designer in a complex setting.
Concept testing
In collaboration with the designers.
To validate our assumptions, we prototyped and tested various Copilot concepts, exploring different ways to integrate AI into the pipeline-building flow with a focus on ease of use, feature visibility, and AI clarity.
User feedback highlighted areas for improvement: some found Copilot’s purpose unclear, suggesting that “a quick guide or onboarding tip might help users understand the AI’s capabilities,” while others expressed a desire for clearer explanations of Copilot’s features.
Branding
In collaboration with designers and marketing team.
To build a strong brand for Copilot, we teamed up with marketing to create a cohesive identity that positioned it as an essential, AI-powered assistant within the Matillion platform.
We designed consistent visual assets like banners and modals to introduce users to Copilot’s capabilities, tailoring messaging to highlight entitlements and encourage upgrades.
Final design
With Copilot’s private preview launch, we introduced an in-context prompt panel, allowing users to build pipelines seamlessly while viewing each component generated in real-time on the canvas space for full control and transparency.
Post-prompt feedback capture supports continuous learning, while users enjoy the flexibility to create from scratch or enhance pipelines intuitively and efficiently.
Design details
We successfully met our deadline to have Copilot demo-ready for the Snowflake Summit, showcasing its capabilities live on stage.
With the launch, we introduced an in-app product-led growth initiative that allows users to upgrade directly within the application to access AI features. This approach not only streamlined the user experience but also provided a direct growth pathway, reinforcing Copilot’s business value by driving adoption and upsell opportunities.
Success metrics
Copilot brought substantial improvements in user experience and efficiency for our customers building data pipelines. Here’s a breakdown of the key success metrics.
Reduce pipeline creation time by 30%.
Average pipeline build time dropped by 25%, just below our target. This suggests Copilot is effective but could be further streamlined.
Adoption rate of 20% in first 3 months.
We reached a 17% adoption rate within the first 3 months, falling short of the target. While feedback was positive, users may need more awareness or training to realise Copilot’s full potential.
60% task completion rate.
42% of users successfully completed pipeline-building tasks using prompts, below our goal. User behaviour points at user using Copilot to experiment and not run completed pipelines.
20% reduction in support content usage.
User visiting support related content to build pipelines decreased
by 15%, just shy of the goal. While Copilot is clearly helping, some complexities still require assistance.
Reduction in onboarding time by 20%.
New users saw a 17% reduction in onboarding time to build their first pipeline, close to the 20% target. Copilot is clearly a helpful tool, but there’s room for further simplification.
Learnings
Overall, we successfully met our core objectives, although some success metrics were highly ambitious given the tight timeline. While we achieved meaningful progress, setting slightly more flexible goals could have allowed us to celebrate incremental wins along the way.
Early user interactions proved invaluable in highlighting real-world usage patterns and unanticipated needs. Conducting interviews and capturing post-prompt feedback gave us direction for iterative improvements and reinforced the value of staying user-centred throughout development.
The compressed timeline highlighted the value of rapid prototyping and prioritising high-impact changes. However, working with time constraints also underscored the need for a clear scope to avoid feature creep, especially with evolving AI technologies.
Next steps
With Copilot still in private preview, we have valuable time to refine the experience before general availability. This phase allows us to address initial user feedback, enhancing usability and feature clarity to ensure a polished, impactful release.
Take a step back to establish a unified AI experience vision that aligns with Matillion’s broader goals. This vision will guide the role of AI across the platform, ensuring consistency, enhancing usability, and helping users understand how AI-driven features can add value across different touch points.