Home主页
Case Study · Azure Machine Learning

Annotation for complex ML pipelines复杂 ML Pipeline 的 Annotation

I designed a lightweight annotation system that helps data scientists explain, revisit, and share complex pipeline graphs—without letting notes take over the canvas. 我为数据科学家设计了一套轻量 Annotation 系统,让复杂 pipeline graph 更容易被解释、回看与协作,同时不让辅助信息干扰主画布。

1 monthfrom ambiguity to MVP design从模糊需求到 MVP 设计
3 scopesnode · multi-node · pipeline节点 · 多节点 · 整条 pipeline
3 principlesefficiency · focus · scalability效率 · 聚焦 · 可扩展
Product designer产品设计师
PM · Engineering · Aether产品 · 工程 · Aether 团队
2021
Azure Machine Learning Designer
AzureML Annotation project overview
01Context背景
The graph showed execution, not intentGraph 展示了执行,却没有解释意图

AzureML users could build and inspect large pipeline graphs, but the graph alone could not explain why a branch existed, what had failed, or what someone should remember before reusing it. People worked around this by adding ordinary nodes as notes. AzureML 用户可以搭建和查看大型 pipeline graph,但 graph 本身无法说明某个分支为何存在、哪里失败过、复用前该注意什么。用户只能把普通 node 当作便签来绕过这个问题。

Product need: reach parity with Aether while creating an experience native to AzureML. 产品需求:补齐与 Aether 的能力差距,同时做出符合 AzureML 语境的体验。
Annotation project constraints
A new feature, no direct usage data, and a one-month timeline.全新功能、缺少直接使用数据、时间只有一个月。
02Insight洞察
Annotations are low-frequency, but high-contextAnnotation 低频,却承载高上下文

Because AzureML had no annotation data yet, I studied real Aether usage and mapped it onto AzureML workflows. Three patterns stood out: AzureML 当时没有 annotation 使用数据,所以我先研究 Aether 的真实用法,再映射到 AzureML workflow。三个模式最明显:

AzureML annotation user scenarios
The scenarios ranged from distinguishing parallel stages to documenting failure reasons.场景从区分并行阶段,到记录失败原因。
03Process过程
Design the workflow before styling the note先设计工作流,再设计便签样式

I mapped when annotations appear across authoring, retraining, and run-detail workflows. That exposed three decisions that mattered more than visual polish. 我先梳理 annotation 在 authoring、retraining 与 run detail 中何时出现。由此明确了三个比视觉样式更关键的决策。

Trigger如何触发Compared right-click, a dedicated tool mode, and context-specific entry points. I chose right-click because it mapped directly to the target and stayed out of the main flow.比较右键、独立工具模式和场景化入口。最终选择右键:它与目标对象直接映射,也不会打断主流程。
Editing如何编辑Compared inline editing, a side panel, and a hybrid. The panel separated rich editing from graph reading and kept the canvas stable.比较 inline、侧边 panel 与混合模式。最终用 panel 承载富文本编辑,让 graph 阅读保持稳定。
Display如何展示Compared indicators, existing parameter patterns, and visible tickets. Direct-on-canvas tickets made context immediately legible, with minimize and hide controls for focus.比较仅图标、沿用 parameter pattern 与直接展示 ticket。最终用 canvas ticket 保留上下文,同时提供最小化与全局隐藏。
AzureML annotation workflow mapping
Mapping the feature to the full AzureML workflow prevented a canvas-only solution.把功能放回完整 AzureML workflow,避免只解决某一个 canvas 状态。
Annotation editing options
Inline speed versus panel depth: the final MVP favored a focused edit panel.Inline 的速度与 panel 的深度之间,MVP 最终选择了专注的编辑 panel。
04Solution方案
Visible when useful, quiet when not需要时可见,不需要时安静

The final system treated annotations as independent, flexible canvas objects. Users could attach them to a node, several nodes, or the whole pipeline; quote targets with #; emphasize key text; resize or minimize a note; and hide all annotations when they needed to read topology. 最终系统把 annotation 定义为独立、灵活的 canvas object。用户可以关联单节点、多节点或整条 pipeline;用 # 引用对象;强调关键信息;调整或最小化便签;需要专注 topology 时也可一键隐藏全部 annotation。

Do not make annotations the main task. Make the main task easier to understand.不要让 Annotation 变成主任务;让它帮助用户更好地理解主任务。
Final AzureML annotation UI
Annotation tickets, rich-text hierarchy, minimize, and a global show/hide control.Annotation ticket、富文本层级、最小化与全局显示 / 隐藏。
05Outcome & reflection结果与反思
User feedback revealed the next product layer用户反馈指出了下一层产品机会

Users valued the visibility of annotations for lineage, cross-platform links, and failed runs. The same feedback also exposed limits in the first version: links needed to render as links, and run-specific annotation deserved first-class support. 用户认可 annotation 在 lineage、跨平台链接和失败任务记录上的可见性。同时,反馈也暴露了首版边界:链接应被真正渲染,特定 run 的 annotation 也值得成为一等能力。

User feedback on AzureML annotations

What I learned我的反思

In an expert tool, clarity often comes from adding context—not adding more interface.在专家工具里,清晰往往来自补充上下文,而不是增加更多界面。