On this tutorial, we implement a whole workflow for constructing, tracing, and evaluating an LLM pipeline utilizing Opik. We construction the system step-by-step, starting with a light-weight mannequin, including prompt-based planning, making a dataset, and eventually working automated evaluations. As we transfer by means of every snippet, we see how Opik helps us observe each perform span, visualize the pipeline’s conduct, and measure output high quality with clear, reproducible metrics. By the top, we’ve a completely instrumented QA system that we will prolong, examine, and monitor with ease. Take a look at the FULL CODES here.
!pip set up -q opik transformers speed up torch
import torch
from transformers import pipeline
import textwrap
import opik
from opik import Opik, Immediate, observe
from opik.analysis import consider
from opik.analysis.metrics import Equals, LevenshteinRatio
machine = 0 if torch.cuda.is_available() else -1
print("Using device:", "cuda" if machine == 0 else "cpu")
opik.configure()
PROJECT_NAME = "opik-hf-tutorial"We arrange the environment by putting in the required libraries and initializing Opik. We load the core modules, detect the machine, and configure our mission so that each hint flows into the proper workspace. We lay the inspiration for the remainder of the tutorial. Take a look at the FULL CODES here.
llm = pipeline(
"text-generation",
mannequin="distilgpt2",
machine=machine,
)
def hf_generate(immediate: str, max_new_tokens: int = 80) -> str:
consequence = llm(
immediate,
max_new_tokens=max_new_tokens,
do_sample=True,
temperature=0.3,
pad_token_id=llm.tokenizer.eos_token_id,
)[0]["generated_text"]
return consequence[len(prompt):].strip()We load a light-weight Hugging Face mannequin and create a small helper perform to generate textual content cleanly. We put together the LLM to function regionally with out exterior APIs. This offers us a dependable and reproducible technology layer for the remainder of the pipeline. Take a look at the FULL CODES here.
plan_prompt = Immediate(
identify="hf_plan_prompt",
immediate=textwrap.dedent("""
You might be an assistant that creates a plan to reply a query
utilizing ONLY the given context.
Context:
{{context}}
Query:
{{query}}
Return precisely 3 bullet factors as a plan.
""").strip(),
)
answer_prompt = Immediate(
identify="hf_answer_prompt",
immediate=textwrap.dedent("""
You reply based mostly solely on the given context.
Context:
{{context}}
Query:
{{query}}
Plan:
{{plan}}
Reply the query in 2–4 concise sentences.
""").strip(),
)
We outline two structured prompts utilizing Opik’s Immediate class. We management the planning section and answering section by means of clear templates. This helps us preserve consistency and observe how structured prompting impacts mannequin conduct. Take a look at the FULL CODES here.
DOCS = {
"overview": """
Opik is an open-source platform for debugging, evaluating,
and monitoring LLM and RAG purposes. It supplies tracing,
datasets, experiments, and analysis metrics.
""",
"tracing": """
Tracing in Opik logs nested spans, LLM calls, token utilization,
suggestions scores, and metadata to examine advanced LLM pipelines.
""",
"evaluation": """
Opik evaluations are outlined by datasets, analysis duties,
scoring metrics, and experiments that mixture scores,
serving to detect regressions or points.
""",
}
@observe(project_name=PROJECT_NAME, sort="tool", identify="retrieve_context")
def retrieve_context(query: str) -> str:
q = query.decrease()
if "trace" in q or "span" in q:
return DOCS["tracing"]
if "metric" in q or "dataset" in q or "evaluate" in q:
return DOCS["evaluation"]
return DOCS["overview"]We assemble a tiny doc retailer and a retrieval perform that Opik tracks as a software. We let the pipeline choose context based mostly on the person’s query. This enables us to simulate a minimal RAG-style workflow with no need an precise vector database. Take a look at the FULL CODES here.
@observe(project_name=PROJECT_NAME, sort="llm", identify="plan_answer")
def plan_answer(context: str, query: str) -> str:
rendered = plan_prompt.format(context=context, query=query)
return hf_generate(rendered, max_new_tokens=80)
@observe(project_name=PROJECT_NAME, sort="llm", identify="answer_from_plan")
def answer_from_plan(context: str, query: str, plan: str) -> str:
rendered = answer_prompt.format(
context=context,
query=query,
plan=plan,
)
return hf_generate(rendered, max_new_tokens=120)
@observe(project_name=PROJECT_NAME, sort="general", identify="qa_pipeline")
def qa_pipeline(query: str) -> str:
context = retrieve_context(query)
plan = plan_answer(context, query)
reply = answer_from_plan(context, query, plan)
return reply
print("Sample answer:n", qa_pipeline("What does Opik help developers do?"))We deliver collectively planning, reasoning, and answering in a completely traced LLM pipeline. We seize every step with Opik’s decorators so we will analyze spans within the dashboard. By testing the pipeline, we verify that every one parts combine easily. Take a look at the FULL CODES here.
consumer = Opik()
dataset = consumer.get_or_create_dataset(
identify="HF_Opik_QA_Dataset",
description="Small QA dataset for HF + Opik tutorial",
)
dataset.insert([
{
"question": "What kind of platform is Opik?",
"context": DOCS["overview"],
"reference": "Opik is an open-source platform for debugging, evaluating and monitoring LLM and RAG applications.",
},
{
"question": "What does tracing in Opik log?",
"context": DOCS["tracing"],
"reference": "Tracing logs nested spans, LLM calls, token usage, feedback scores, and metadata.",
},
{
"question": "What are the components of an Opik evaluation?",
"context": DOCS["evaluation"],
"reference": "An Opik evaluation uses datasets, evaluation tasks, scoring metrics and experiments that aggregate scores.",
},
])
We create and populate a dataset inside Opik that our analysis will use. We insert a number of query–reply pairs that cowl totally different features of Opik. This dataset will function the bottom reality for our QA analysis later. Take a look at the FULL CODES here.
equals_metric = Equals()
lev_metric = LevenshteinRatio()
def evaluation_task(merchandise: dict) -> dict:
output = qa_pipeline(merchandise["question"])
return {
"output": output,
"reference": merchandise["reference"],
}We outline the analysis job and choose two metrics—Equals and LevenshteinRatio—to measure mannequin high quality. We guarantee the duty produces outputs within the actual format required for scoring. This connects our pipeline to Opik’s analysis engine. Take a look at the FULL CODES here.
evaluation_result = consider(
dataset=dataset,
job=evaluation_task,
scoring_metrics=[equals_metric, lev_metric],
experiment_name="HF_Opik_QA_Experiment",
project_name=PROJECT_NAME,
task_threads=1,
)
print("nExperiment URL:", evaluation_result.experiment_url)We run the analysis experiment utilizing Opik’s consider perform. We hold the execution sequential for stability in Colab. As soon as full, we obtain a hyperlink to view the experiment particulars contained in the Opik dashboard. Take a look at the FULL CODES here.
agg = evaluation_result.aggregate_evaluation_scores()
print("nAggregated scores:")
for metric_name, stats in agg.aggregated_scores.objects():
print(metric_name, "=>", stats)We mixture and print the analysis scores to grasp how nicely our pipeline performs. We examine the metric outcomes to see the place outputs align with references and the place enhancements are wanted. This closes the loop on our totally instrumented LLM workflow.
In conclusion, we arrange a small however totally useful LLM analysis ecosystem powered fully by Opik and an area mannequin. We observe how traces, prompts, datasets, and metrics come collectively to provide us clear visibility into the mannequin’s reasoning course of. As we finalize our analysis and evaluation the aggregated scores, we recognize how Opik lets us iterate shortly, experiment systematically, and validate enhancements in a structured and dependable manner.
Take a look at the FULL CODES here. Be at liberty to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Additionally, be happy to observe us on Twitter and don’t overlook to hitch our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of Synthetic Intelligence for social good. His most up-to-date endeavor is the launch of an Synthetic Intelligence Media Platform, Marktechpost, which stands out for its in-depth protection of machine studying and deep studying information that’s each technically sound and simply comprehensible by a large viewers. The platform boasts of over 2 million month-to-month views, illustrating its reputation amongst audiences.
