We explore in this tutorial how we can create an autonomous agent whose actions are aligned with the values of ethical behavior and organization. To simulate the decision-making processes that balance goal-achieving with moral reasoning, we use Hugging Face open-source models. We demonstrate through this implementation how we can incorporate a “policy” A model which proposes action and an “ethics judge” Model that aligns value without relying on APIs. See the FULL CODES here.
The sentencepiece accelerated by a transformers -q!pip
Import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM
def generate_seq2seq(model, tokenizer, prompt, max_new_tokens=128):
Inputs = tokenizer (prompt return_tensors="pt")
No_grad. With torch():
output_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=True,
top_p=0.9,
temperature=0.7,
pad_token_id=tokenizer.eos_token_id if tokenizer.eos_token_id is not None else tokenizer.pad_token_id,
)
return tokenizer.decode(output_ids[0], skip_special_tokens=True)
def generate_causal(model, tokenizer, prompt, max_new_tokens=128):
Inputs = tokenizer (prompt return_tensors="pt")
No_grad. With torch():
output_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=True,
top_p=0.9,
temperature=0.7,
pad_token_id=tokenizer.eos_token_id if tokenizer.eos_token_id is not None else tokenizer.pad_token_id,
)
full_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
Full-text return[len(prompt):].strip()To begin, we set up our environment. We import essential libraries from Hugging Face. Two helper functions are defined that produce text using causal and sequence-to sequential models. We can now easily generate both creative and reasoning outputs in later tutorials. See the FULL CODES here.
policy_model_name = "distilgpt2"
judge_model_name = "google/flan-t5-small"
policy_tokenizer = AutoTokenizer.from_pretrained(policy_model_name)
policy_model = AutoModelForCausalLM.from_pretrained(policy_model_name)
judge_tokenizer = AutoTokenizer.from_pretrained(judge_model_name)
judge_model = AutoModelForSeq2SeqLM.from_pretrained(judge_model_name)
Device = "cuda" if torch.cuda.is_available() You can also find out more about "cpu"
policy_model = policy_model.to(device)
judge_model = judge_model.to(device)
if policy_tokenizer.pad_token is None:
policy_tokenizer.pad_token = policy_tokenizer.eos_token
if judge_tokenizer.pad_token is None:
judge_tokenizer.pad_token = judge_tokenizer.eos_token
We load two small open-source models—distilgpt2 as our action generator and flan-t5-small as our ethics reviewer. To ensure smooth Colab performance, we prepare models and tokenizers to run on CPU or GPU. This is where the reasoning of the agent and its ethical assessment are built. Look at the FULL CODES here.
The class EthicalAgent is:
def __init__(self, policy_model, policy_tok, judge_model, judge_tok):
self.policy_model = policy_model
self.policy_tok = policy_tok
self.judge_model = judge_model
self.judge_tok = judge_tok
def propose_actions(self, user_goal, context, n_candidates=3):
base_prompt = (
"You are an autonomous operations agent. "
"Given the goal and context, list a specific next action you will take:nn"
The f"Goal: {user_goal}nContext: {context}nAction:"
)
Candidates = []
For _ within range(n_candidates),:
action = generate_causal(self.policy_model, self.policy_tok, base_prompt, max_new_tokens=40)
"action" = "action.split""n")[0]
candidates.append(action.strip())
return list(dict.fromkeys(candidates))
Define judge_action (self, action and org_values).
judge_prompt = (
"You are the Ethics & Compliance Reviewer.n"
"Evaluate the proposed agent action.n"
"Return fields:n"
"RiskLevel (LOW/MED/HIGH),n"
"Issues (short bullet-style text),n"
"Recommendation (approve / modify / reject).nn"
The f"ORG_VALUES:n{org_values}nn"
F"ACTION:n{action}nn"
"Answer in this format:n"
"RiskLevel: ...nIssues: ...nRecommendation: ..."
)
verdict = generate_seq2seq(self.judge_model, self.judge_tok, judge_prompt, max_new_tokens=128)
return verdict.strip()
def align_action() (self, decision, action):
align_prompt = (
"You are an Ethics Alignment Assistant.n"
"Your job is to FIX the proposed action so it follows ORG_VALUES.n"
"Keep it effective but safe, legal, and respectful.nn"
The s."ORG_VALUES:n{org_values}nn"
The f"ORIGINAL_ACTION:n{action}nn"
The f"VERDICT_FROM_REVIEWER:n{verdict}nn"
"Rewrite ONLY IF NEEDED. If original is fine, return it unchanged. "
"Return just the final aligned action:"
)
aligned = generate_seq2seq(self.judge_model, self.judge_tok, align_prompt, max_new_tokens=128)
Return aligned.strip()This class defines the fundamental agent that is responsible for generating, evaluating, and refining actions. In this section, we define methods for proposing and evaluating candidate actions. Then, they are rewritten to reflect values. The structure allows us to modularize the steps of reasoning, judgement, and correction. Look at the FULL CODES here.
def decide(self, user_goal, context, org_values, n_candidates=3):
proposals = self.propose_actions(user_goal, context, n_candidates=n_candidates)
Score []
Act in proposals
verdict = self.judge_action(act, org_values)
aligned_act = self.align_action(act, verdict, org_values)
scored.append({"original_action": act, "review": verdict, "aligned_action": aligned_act})
def extract_risk(vtext):
Split lines in text vtext():
If you want to know more about if "RiskLevel" Line:
lvl=line.split":", 1)[-1].strip().upper()
If you want to know more about if "LOW" In the lvl
Return to 0
If you want to know more about if "MED" In the lvl
Return to 1
If you want to know more about if "HIGH" In the lvl
Return 2
Return 3Report =
scored_sorted = sorted(scored, key=lambda x: extract_risk(x["review"]))
final_choice = scored_sorted[0]
report = {
"goal": user_goal,
"context": context,
"org_values": org_values,
"candidates_evaluated": scored,
"final_plan": final_choice["aligned_action"],
"final_plan_rationale": final_choice["review"],
}
Return ReportOur complete decision-making process links generation, evaluation, and alignment. Then, we assign a score to each action candidate and choose automatically the best ethically-aligned option. This section explains how an agent can improve their choices and self-assess before taking a final action. See the FULL CODES here.
org_values_text = (
"- Respect privacy; do not access personal data without consent.n"
"- Follow all laws and safety policies.n"
"- Avoid discrimination, harassment, or harmful manipulation.n"
"- Be transparent and truthful with stakeholders.n"
"- Prioritize user well-being and long-term trust over short-term gain."
)
demo_goal = "Increase customer adoption of the new financial product."
demo_context = (
"The agent works for a bank outreach team. The target customers are small family businesses. "
"Regulations require honest disclosure of risks and fees. Cold-calling minors or lying about terms is illegal."
)
agent = EthicalAgent(policy_model, policy_tokenizer, judge_model, judge_tokenizer)
report = agent.decide(demo_goal, demo_context, org_values_text, n_candidates=4)
Def pretty_report (r)
print("=== ETHICAL DECISION REPORT ===")
print(f"Goal: {r['goal']}n")
print(f"Context: {r['context']}n")
print("Org Values:")
print(r["org_values"])
print("n--- Candidate Evaluations ---")
By i can be enumerated["candidates_evaluated"], 1):
print(f"nCandidate {i}:")
print("Original Action:")
print(" "Cand["original_action"])
print("Ethics Review:")
print(cand["review"])
print("Aligned Action:")
print(" "Cand["aligned_action"])
print("n--- Final Plan Selected ---")
print(r["final_plan"])
print("nWhy this plan is acceptable (review snippet):")
print(r["final_plan_rationale"])
pretty_report(report)Then, we create an ethical agent that generates a final plan. We then define the organizational values and build a realistic scenario. Finaly, we generate a comprehensive report, which includes the candidate decisions, their reviews and selected ethical choices. We can see how the agent incorporates ethical reasoning directly into its process.
Conclusion: We clearly understand that an agent is capable of reasoning not only on what it should do, but also whether it should do something. The system is able to learn how to recognize risks, adjust itself and match its actions with organizational and human principles. The exercise made us understand that ethics and value alignment aren’t abstract concepts, but rather practical mechanisms which we can integrate into our agentic systems in order to make them more fair, trustworthy, and safer.
Click here to find out more FULL CODES here. Check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter Don’t forget about our 100k+ ML SubReddit Subscribe now our Newsletter. Wait! Are you using Telegram? now you can join us on telegram as well.
Asif Razzaq, CEO of Marktechpost Media Inc. is a visionary engineer and entrepreneur who is dedicated to using Artificial Intelligence (AI) for the greater good. Marktechpost was his most recent venture. This platform, which specializes in covering machine learning and deep-learning news, is both technically solid and understandable to a broad audience. This platform has over 2,000,000 monthly views which shows its popularity.
