Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
  • DailyAI Briefing: A Lean-checked Riemann proof, plus 4 extra
AI-trends.todayAI-trends.today
Home»Tech»Learn how to build an agentic deep reinforcement learning system with curriculum progression, adaptive exploration, and meta-level UCB planning.

Learn how to build an agentic deep reinforcement learning system with curriculum progression, adaptive exploration, and meta-level UCB planning.

Tech By Gavin Wallace19/11/20256 Mins Read
Facebook Twitter LinkedIn Email
Can LLMs Really Judge with Reasoning? Microsoft and Tsinghua Researchers
Can LLMs Really Judge with Reasoning? Microsoft and Tsinghua Researchers
Share
Facebook Twitter LinkedIn Email

We build a Deep Reinforcement System that is advanced and agentic. It guides the agent in learning not just actions but how to pick its own strategies. Designing a Dueling Double DQN Learner. Introducing a curriculum that increases in difficulty. Integrating multiple exploration mode adapts to the training. The meta-agent we build plans, regulates and evaluates the whole learning process. It allows us to understand how agency turns reinforcement learning from a strategic, self-directed workflow into a self directed, autonomous one. Click here to see the FULL CODES here.

!pip install -q gymnasium[classic-control] The torch matplotlib


Import gymnasium as Gym
Numpy can be imported as np
Import torch as torch.nn.
Import deque defaultdict
Import matplotlib.pyplot, math random as Plt


random.seed(0); np.random.seed(0); torch.manual_seed(0)


class DuelingQNet(nn.Module):
   def __init__(self, obs_dim, act_dim):
 Supermarkets are a great way to buy goods and services.().__init__()
       hidden = 128
       self.feature = nn.Sequential(
           nn.Linear(obs_dim, hidden),
           nn.ReLU(),
       )
       self.value_head = nn.Sequential(
           nn.Linear(hidden, hidden),
           nn.ReLU(),
           nn.Linear(hidden, 1),
       )
       self.adv_head = nn.Sequential(
           nn.Linear(hidden, hidden),
           nn.ReLU(),
           nn.Linear(hidden, act_dim),
       )


   def forward(self, x):
 Self.feature(x), h
       v = self.value_head(h)
 A = self.adv_head.h
       return v + (a - a.mean(dim=1, keepdim=True))


class ReplayBuffer:
   def __init__(self, capacity=100000):
       self.buffer = deque(maxlen=capacity)
   def push(self, s,a,r,ns,d):
       self.buffer.append((s,a,r,ns,d))
   def sample(self, batch_size):
       batch = random.sample(self.buffer, batch_size)
       s,a,r,ns,d = zip(*batch)
 Define to_t (x,dt) and return the torch.tensor.
       return to_t(s,torch.float32), to_t(a,torch.long), to_t(r,torch.float32), to_t(ns,torch.float32), to_t(d,torch.float32)
   def __len__(self): return len(self.buffer)

Set up our core system of deep reinforcement learning. Initiate the environment. Create the Dueling Q-Network. Prepare the Replay Buffer to effectively store transitions. While we are establishing these foundations, our agent is equipped with everything it needs to learn. Click here to view the FULL CODES here.

Class DQNAgent
   def __init__(self, obs_dim, act_dim, gamma=0.99, lr=1e-3, batch_size=64):
       self.q = DuelingQNet(obs_dim, act_dim).to(device)
       self.tgt = DuelingQNet(obs_dim, act_dim).to(device)
       self.tgt.load_state_dict(self.q.state_dict())
       self.buf = ReplayBuffer()
       self.opt = optim.Adam(self.q.parameters(), lr=lr)
       self.gamma = gamma
       self.batch_size = batch_size
       self.global_step = 0


   def _eps_value(self, step, start=1.0, end=0.05, decay=8000):
       return end + (start - end) * math.exp(-step/decay)


 Select_action (self, mode, state, strategy, softmax_temp=1.0, def)
       s = torch.tensor(state, dtype=torch.float32, device=device).unsqueeze(0)
 No_grad. With torch():
           q_vals = self.q(s).cpu().numpy()[0]
 if mode == "eval":
           return int(np.argmax(q_vals)), None
       if strategy == "epsilon":
           eps = self._eps_value(self.global_step)
 If random() 

The agent must be able to observe the surrounding environment and choose actions. It also needs a neural network that can update itself. Double DQN, gradient updates and exploration strategies are implemented to balance the learning and discovering of our agent. After we’ve finished this sample, our agent will have its low-level intelligence capabilities. Click here to view the FULL CODES here.

Class MetaAgent
   def __init__(self, agent):
 Self-agent = Agent
       self.levels = {
           "EASY": 100,
           "MEDIUM": 300,
           "HARD": 500,
       }
       self.plans = []
 For diff. in self.levels.keys():
 For mode ["train", "eval"]:
 For expl. ["epsilon", "softmax"]:
                   self.plans.append((diff, mode, expl))
       self.counts = defaultdict(int)
       self.values = defaultdict(float)
       self.t = 0
       self.history = []


   def _ucb_score(self, plan, c=2.0):
 Self.counts[plan]
 n == 0.
 Return float ("inf")
 Self-values[plan] + c * math.sqrt(math.log(self.t+1) / n)


   def select_plan(self):
       self.t += 1
       scores = [self._ucb_score(p) for p in self.plans]
 Return self-plans[int(np.argmax(scores))]


   def make_env(self, diff):
       max_steps = self.levels[diff]
       return gym.make("CartPole-v1", max_episode_steps=max_steps)


   def meta_reward_fn(self, diff, mode, avg_return):
       r = avg_return
       if diff == "MEDIUM"The following equation is equivalent to a 20-r
       if diff == "HARD"R = 50
 If mode == "eval" == "HARD"R = 50
 Return r


   def update_plan_value(self, plan, meta_reward):
       self.counts[plan] += 1
 n ='self.counts[plan]
 m = values of oneself[plan]
       self.values[plan] = (meta_reward-mu) + mu / n


   def run(self, meta_rounds=30):
       eval_log = {"EASY":[], "MEDIUM":[], "HARD":[]}
 If k is in the range (meta_rounds+1), then:
 diff. mode expl = self.select_plan()
 Self.make_env()
           avg_ret = self.agent.run_episodes(env, 5 if mode=="train" Otherwise 3, mode, if mode=="train" You can also find out more about "epsilon")
           env.close()
           if k % 3 == 0:
               self.agent.update_target()
           meta_r = self.meta_reward_fn(diff, mode, avg_ret)
           self.update_plan_value((diff,mode,expl), meta_r)
           self.history.append((k, diff, mode, expl, avg_ret, meta_r))
 If mode == "eval":
               eval_log[diff].append((k, avg_ret))
           print(f"{k} {diff} {mode} {expl} {avg_ret:.1f} {meta_r:.1f}")
       return eval_log

Design the agentic layers that determine how an agent will train. We then use a UCB to choose difficulty levels, modes and exploration style based on previous performance. The meta-agent is guiding our training by strategically selecting the options. See the FULL CODES here.

tmp_env = gym.make("CartPole-v1", max_episode_steps=100)
obs_dim, act_dim = tmp_env.observation_space.shape[0], tmp_env.action_space.n
tmp_env.close()


agent = DQNAgent(obs_dim, act_dim)
MetaAgent = Agent


eval_log = meta.run(meta_rounds=36)


final_scores = agent.evaluate_across_levels(meta.levels, episodes=10)
print("Final Evaluation")
For k, V in final_scores.items():
   print(k, v)

The meta-rounds bring it all together. In these rounds, the DQN agent selects and executes plans that the meta-agent has selected. As the performance of our agent evolves, we can track its adaptation to more challenging tasks. This snippet shows the rise of self-directed long-term learning. See the FULL CODES here.

plt.figure(figsize=(9,4))
Colors are different. [("EASY","tab:blue"), ("MEDIUM","tab:orange"), ("HARD","tab:red")]:
 if eval_log[diff]:
       x, y = zip(*eval_log[diff])
       plt.plot(x, y, marker="o", label=f"{diff}")
plt.xlabel("Meta-Round")
plt.ylabel("Avg Return")
plt.title("Agentic Meta-Control Evaluation")
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

In the graphs, we can see how an agent’s performance changes over time for Easy, Medium and Hard tasks. The curves allow us to observe trends in learning, progress, and agentic planning. We gain insights into the strategic choices that influence an agent’s progress as we examine these plots.

As a conclusion, we see our agent evolving into a multi-level system, one that refines policies, adapts exploration and chooses strategically how to train. We see the meta-agent improve its planning through UCB and lead the lower-level agent to more complex tasks. By understanding how agents can amplify the effects of reinforcement learning we are able to create systems which plan, adjust, and optimise their improvement over time.


Click here to find out more FULL CODES here. Please feel free to browse our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter Don’t forget about our 100k+ ML SubReddit Subscribe now our Newsletter. Wait! What? now you can join us on telegram as well.


Asif Razzaq, CEO of Marktechpost Media Inc. is a visionary engineer and entrepreneur who is dedicated to harnessing Artificial Intelligence’s potential for the social good. Marktechpost is his latest venture, a media platform that focuses on Artificial Intelligence. It is known for providing in-depth news coverage about machine learning, deep learning, and other topics. The content is technically accurate and easy to understand by an audience of all backgrounds. Over 2 million views per month are a testament to the platform’s popularity.

🙌 Follow MARKTECHPOST: Add us as a preferred source on Google.

ada ar learning met meta stem x
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

Melania Trump’s AI Era is Here

The AI Security Debate Has Misplaced the Plot. We Requested 10 Specialists What the Actual Nightmares Are

‘Uncanny Valley’: Pentagon vs. ‘Woke’ Anthropic, Agentic vs. Mimetic, and Trump vs. State of the Union

Vibe Coding Is the New Open Source—in the Worst Way Possible

Muse Marvel Places Meta Again on the AI Frontier

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

You might be surprised at how closely the US and China collaborate on AI.

21/01/2026

The Worst First Job You Can Give an Agent Is the Visible One – Unite.AI

03/08/2026
Latest News

Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI

22/11/2026

OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.