I must inform you, the group behind AgentOps is working nice.
After I was studying about their options, I used to be stunned by the best way they suppose and have carried out these insane options for customers.
Listed below are a number of the finest ones:
a) Auto-Instrumentation of LLM calls: After initializing AgentOps, the SDK robotically detects and devices calls to put in LLM (giant language mannequin) suppliers like GPT, Claude, Gemini, and others.
This automation permits builders to trace each interplay visually between their utility and the LLM supplier with out extra code.
b) Complete session administration: AgentOps organizes each interplay and occasion inside a session, encapsulating all actions, LLM calls, and agent actions beneath one execution occasion.
After which every session information attributes like session ID, begin and finish timestamps, success or failure standing, and optionally available tags for simpler categorization and retrieval.
Additionally, this device helps each single-session and multi-session modes, permitting asynchronous administration and seamless scalability for complicated initiatives.
c) Session Waterfall dashboard: They even present a characteristic known as Session Waterfall on the AgentOps dashboard which provides a timeline view of each LLM name, motion, and error in a session.
It’s particularly helpful for debugging — customers can see precisely when every motion occurred, assessment the precise inputs and outputs, and establish failures at a look.
d) Error dealing with and logging: AgentOps may be useful to seize errors throughout all occasion sorts, enabling exact debugging.
Every error is logged with particular particulars like error kind, error code, timestamp, and stack hint.
For every ErrorEvent, customers can see the triggering occasion, making it simple to hint the error’s origin and resolve it effectively.
Effectively, they’ve extra such helpful options, you’ll be able to learn extra on the docs.