Widget Enrichment

Why Enrichment?

When an agent generates a widget payload, you often need additional data that the LLM shouldn't generate:

• Real-time data: Inventory levels, distances, pricing, availability
• Large datasets: Full vendor details, product catalogs, user history
• Computed values: Distances, ETAs, personalized recommendations

Without enrichment, you'd have to either:

1. Have the LLM generate this data (wastes tokens, introduces hallucination risk)
2. Make the widget fetch data client-side (causes loading flicker, requires client auth)

Enrichment solves this by fetching data during response generation, so widgets render with complete data immediately.

Server vs Client Enrichment

There are two approaches to enrichment:

Rule of thumb: Use server enrichment for data the widget needs to function. Use client enrichment for visual enhancements that can load progressively.

Server Enrichment

Server enrichment runs parallel to LLM generation. The results are saved with the message and passed to your render() function.

Basic Example

createWidget({
  widgetType: 'vendor-list',
  schema: z.object({
    // LLM returns just IDs to save tokens
    vendorIds: z.array(z.string()),
  }),
  reactDOM: ReactDOM,
  enrich: {
    // Call the get-vendor-details tool with vendorIds as input
    details: {
      toolId: 'get-vendor-details',
      inputs: {
        ids: "${vendorIds|join(',')}",  // Template from payload
      },
    },
  },
  render: (payload, enriched) => (
    <VendorList
      ids={payload.vendorIds}
      details={enriched.details}  // Tool result
    />
  ),
})

How It Works

1. LLM generates widget payload (e.g., { vendorIds: ['v1', 'v2', 'v3'] })
2. Server calls your tool with the templated inputs
3. Tool result is attached to the message
4. render() receives both payload and enriched data

Multiple Enrichments

You can call multiple tools in parallel:

enrich: {
  details: {
    toolId: 'get-vendor-details',
    inputs: { ids: "${vendorIds|join(',')}" },
  },
  ratings: {
    toolId: 'get-vendor-ratings',
    inputs: { vendorIds: "${vendorIds}" },
  },
},
render: (payload, enriched) => (
  <VendorList
    ids={payload.vendorIds}
    details={enriched.details}
    ratings={enriched.ratings}
  />
),

Input Templating

Use ${fieldName} to reference payload fields in tool inputs:

• ${vendorIds} - Direct value
• ${vendorIds|join(',')} - Join array with delimiter
• ${origin.lat} - Nested field access

Client Enrichment

Client enrichment fetches data after the widget renders, using the invokeTool function provided to your render function.

Basic Example

createWidget({
  widgetType: 'vendor-list',
  schema: z.object({
    vendorIds: z.array(z.string()),
    vendors: z.array(z.object({
      id: z.string(),
      name: z.string(),
      location: z.object({ lat: z.number(), lng: z.number() }),
    })),
  }),
  reactDOM: ReactDOM,
  // Server enrichment for vendor details
  enrich: {
    details: {
      toolId: 'get-vendor-details',
      inputs: { ids: "${vendorIds|join(',')}" },
    },
  },
  render: (payload, enriched, { invokeTool }) => (
    <VendorListWithDistance
      vendors={enriched.details}
      invokeTool={invokeTool}  // Pass to component for client enrichment
    />
  ),
})

function VendorListWithDistance({ vendors, invokeTool }) {
  const [distances, setDistances] = useState<Record<string, string>>({});

  useEffect(() => {
    // Fetch distances client-side (requires user's location)
    navigator.geolocation.getCurrentPosition(async (pos) => {
      const result = await invokeTool('distance-matrix', {
        args: {
          origin: { lat: pos.coords.latitude, lng: pos.coords.longitude },
          destinations: vendors.map(v => v.location),
        },
      });
      if (result.success) {
        setDistances(result.data);
      }
    });
  }, [vendors, invokeTool]);

  return (
    <div>
      {vendors.map(v => (
        <VendorCard
          key={v.id}
          vendor={v}
          distance={distances[v.id]}  // May be undefined initially
        />
      ))}
    </div>
  );
}

Client Enrichment Limitations

Client enrichment has important limitations you should understand:

1. Not Saved to Message History

Client-enriched data is fetched live each time the widget renders. It's not saved with the message. This means:

• Refreshing the page re-fetches the data
• Data may differ between views if underlying data changed
• Historical conversations show current data, not data at conversation time

2. Not Visible in Feedback or Tracing

When users leave feedback or admins review message traces, client-enriched data is not shown. The feedback view displays:

• The original LLM payload ✓
• Server-enriched data ✓
• Client-enriched data ✗

This is a framework-level limitation—the platform cannot capture arbitrary client-side fetches for generic widgets.

3. Should Not Break the Widget

Design your widget so it works without client-enriched data. The data should be a visual enhancement, not a functional requirement:

// Good: Distance is optional enhancement
<VendorCard
  vendor={v}
  distance={distances[v.id]}  // Shows "Calculating..." or nothing if undefined
/>

// Bad: Widget breaks without distance
<VendorCard
  vendor={v}
  distance={distances[v.id]!}  // Crashes if undefined
  onClick={() => sortByDistance(distances)}  // Fails without data
/>

When to Use Client Enrichment

Use client enrichment for:

• User-location-dependent data (distances, local time)
• Data that changes frequently and should always be fresh
• Progressive enhancements that improve UX but aren't critical
• Data that requires client-side context (geolocation, device info)

Use server enrichment for:

• Data the widget needs to function
• Data that should be preserved in history
• Data that needs to appear in feedback/tracing
• Data that doesn't depend on client context