Modeling conversations

Model chat as an ordered thread, scope it to the person or account it belongs to, and assemble a bounded mix of recent and relevant turns for each prompt.

The three identities

A useful conversation model separates three jobs:

The same entity can have many threads. For example, one customer can open several support tickets while semantic retrieval still finds an earlier resolution belonging to that customer. Prefer one entity per Memory.Thread conversation. A raw thread can contain several entities, but the server selects the last N turns before the helper filters to its Memory entity. A mixed-entity helper thread can therefore return fewer than N recent items and will not scan farther back to fill the window.

Support handoff

Create one Memory for the customer and one Thread for the ticket. Append each message as it arrives, then ask context for the recent exchange plus relevant earlier turns before an agent or model responds.

memory = Memory("customer-42", client=client)
thread = memory.thread("ticket-8492")

thread.append(
    "Sign-in still fails after the password reset.",
    {"speaker": "customer", "channel": "email"},
)
thread.append(
    "We verified the reset link and are checking the account lock.",
    {"speaker": "agent", "channel": "email"},
)

context = thread.context(
    "What has already been tried, and what should the next agent check?",
    last_n_turns=8,
)

An accepted append can briefly precede local search projection. The SDK transport already retries the origin's 503 thread_projection_pending response, honoring Retry-After within its configured bounded retry budget. If context still exhausts that budget, apply application-level bounded backoff; reuse the original append idempotency key when retrying the write, and never create a fresh logical turn just to wait for projection.

context returns one flat list of Memory items. The latest ordered window comes first; up to five older semantic matches follow. A document that appears in both sets is returned once, with the ordered turn winning. This makes it straightforward to format item.text values into a prompt while keeping the amount of transcript data bounded.

The helper downloads at most eight turns concurrently and shares one 16 MiB body-byte budget across recent turns and non-duplicate semantic matches. Once the budget is crossed it schedules no later batch and returns the SDK's ordinary local error. Keep a smaller application-specific token budget as well; byte limits do not predict every model's tokenization.

For durable workers that may restart after sending a request, use the raw append method and persist your own idempotency key with the message. The helper protects retries within one SDK call; a caller-owned key lets a later process resume the same logical append safely.

Therapy-oriented session context

The same model can assemble context for a conversational wellbeing application: the patient is the entity, while one longitudinal thread carries the ordered history that later sessions need to reference. Record the appointment number in metadata rather than starting a new thread for every appointment. Here, session 4 asks for context that includes a strategy recorded in session 3.

Assume clinic_client / clinicClient is already scoped to the clinic's partition and the current care-team principal. Regulated turns need an explicit reader list, so this example uses the raw append method instead of Memory.Thread.append. It then uses the entity-scoped helper only to assemble context.

clinic_client.append_thread(
    "patient-7-care",
    "I used the breathing exercise before the flight and felt calmer.",
    entity_id="patient-7",
    metadata={"speaker": "patient", "session_number": 3},
    acl_readers=["group:care-team-7"],
    idempotency_key="patient-7-session-3-turn-4",
)

patient = Memory("patient-7", client=clinic_client)
history = patient.thread("patient-7-care")
context = history.context(
    "At session 4, which coping strategies did the patient find helpful earlier?",
    last_n_turns=6,
)

Because session 3 and session 4 share patient-7-care, the helper's ordered window and same-thread semantic search can include the earlier strategy. If each appointment must be a separate thread, compose that thread's recent context with a separate entity-wide Memory.recall instead; same-thread context cannot cross into another thread.

Use a partition per end-client when one service hosts several clinics or practices. Thread reads and semantic retrieval enforce both that hard boundary and reader lists. Reader-list state has three distinct meanings: omitted is tenant-visible within the scope, a non-empty list restricts reads to matching principals (plus audited admin bypass), and an explicit empty list is admin-only quarantine. Memory.Thread.append omits the list and is therefore tenant-visible; use raw append, as above, when a conversation needs reader labels.

Keep one reader policy across a thread when sequence membership is sensitive. Reader filtering never returns denied content, but canonical turn_index values are not renumbered, so mixing reader lists can reveal that an unreadable position exists. Split conversations into separate threads when principals should not learn about one another's turns.

Shape context deliberately

• Keep last_n_turns small enough for your prompt budget. The default is 10; valid values are 1 through 1000.
• Leave recent_first false for normal chronological dialogue. Set it true only when the consumer explicitly wants newest-first recent turns.
• Treat semantic matches as supporting history, not as a replacement for the canonical ordered window. They follow the recent turns and can come from earlier in the same thread.
• Store speaker, channel, model, or session kind as scalar metadata. Do not hide ordering or ownership in tags: thread_id, turn_index, and entity_id are first-class fields.
• Use the same partition-scoped client for Memory and Thread so append, read, download, and semantic search share one hard boundary.

For lower-level request fields, status codes, and idempotency behavior, see the Conversational Threads API.