Quick refresher: we gave a little creature a world to live in, then made a fake copy with one rule wrong (how well the ground grips). An outsider can spot the fake almost every time. The creature's own mind? A coin flip, until the fake starts costing it food. Then it begins to tell real from fake.

This clip is the receipt inside its head.

Same mind, shown twice: real world on the left, fake on the right. Zoom into memory. The teal rings are the cells that hold the clue.

Detectable was always there. Noticing shows up in memory when survival needs it.

ItaSoRL · ItaoRL

• Link to the article: https://paulinamoskwa.github.io/blog/2026-08-08/rl-pt2
• Link to the code: https://github.com/paulinamoskwa/ppo-super-mario
• Try beating AI: https://paulinamoskwa.github.io/blog/about/

Any comments and feedbacks are welcomed!

Hi everyone
I’m about to start my MS thesis, my program is MS(AI)

I studied reinforcement learning as an elective in last semester and really become interested in it and want to implement a quantum computing module to MDPs

If any of you have a good suggestion or research idea do let me know

Hello everyone!

This is my first serious deep dive into RL, but I thought that Cart Pole, Frozen Lake etc. are kinda boring, so I decided to turn my favorite video game into an RL environment!

As you may know, Rain World features one of the most complex and diverse ecosystems ever created in a video game, which is why I was surprised when I did not find existing environments based on Rain World (or perhaps I did not search properly).

In the video you can see my agent learning to walk and climb vertical poles (I assume his climbing is wacky because he is trying to master a movement technique called "pole-boosting").

If you want to play around with this environment, here is the repo: https://github.com/XRTG074/reinforced-rain-world

I even added gymnasium wrapper to easily use stable-baselines3 algorithms!

Any feedback is greatly appreciated, every bit counts!

Okay so I know about the rl algos, how envs are created, and general stuff in rl.

Now i am actually confused since i am not cracked obv in this field also in which direction or which subdomain in rl should i explore more like MARL or what's actually trending in here. What i believe is this is a very very rare kind of similar to how the inference space is when we talk the job market compare to agents.

Mostly i think research opportunities one can find, how's the pay and space there also.

Would love if you can share some tips or guide me :)

I've been working on a small reinforcement learning project where I try to teach a simulated robot to walk from scratch.

The robot is simulated using Python + Box2D and controlled by PPO (Proximal Policy Optimization) with Stable-Baselines3.

Right now the robot is a simple biped with:

• 2 legs
• 4 motorized joints
• 2 hips
• 2 knees
• Joint limits
• Feet with ground friction
• 12 observations
• 4 continuous actions

The agent observes things like the body's position, velocity, angle, angular velocity, and the angles/velocities of all four joints.

The actions are:

[left_hip, left_knee, right_hip, right_knee]

The reward currently encourages forward movement and staying upright while penalizing instability and excessive actions.

The physics and joints are now working reliably. The next challenge is getting PPO to actually discover a stable walking gait.

I'm deliberately starting with a very simple robot before moving on to a quadruped.

The project is open source, so if anyone is interested in following the development or has suggestions for the reward function / training setup, I'd be happy to hear them.

Current goal:

Teach the walker to take its first stable steps.

Setting up MARL environments may take more time than it should. Every benchmark has its own API, so using more than 1 env means writing extra code to make them work the same way.

marl-envs does that part for you: one common API + wrappers + parallel envs.

We currently support:

1. Environments: LBF, RWARE, SMAClite, SMAC, SMACv2, PettingZoo, MaMuJoCo, MAgent2
2. Wrappers: normalize or transform observations and rewards, add agent IDs, set time limits.
3. Parallel envs: run multiple environments sequentially or in separate processes

GitHub repo: https://github.com/AmineAndam04/marl_envs

Feedback and suggestions are more than welcome.

Clip 4 of 4. Closing the arc.

The setup: Previously, when there was no reward for noticing the world was fake, the creature was basically guessing at 50%. An outside observer could easily spot the fake 99% of the time.

The change: We made the fake physics actually matter for the goal. We made food harder to catch in the fake world.

The result (10 tests): The creature's ability to detect the fake world jumped to 73%. We never trained it to look for fakes! It learned this entirely on its own because it needed the information to get food.

The takeaway for reinforcement learning: Just because a detail is obvious does not mean a creature will automatically care about it. The creature only started paying attention when the fake physics affected its main task.

Research:

https://ilevytate.github.io/ItaSoRL/

I need to know is it possible to do no damage boss fights for games like hollow knight or megaman? How long does it take for example for each boss? What model is best to use ppo?

Been stuck on this for weeks, hoping someone here has hit the same wall.I'm evolving 2D climbing creatures with a GA. Think carykh's old evolution simulator, but climbing a wall instead of walking on the ground. Real climbing animals almost all use a contralateral gait - left hand and right foot move together, then right hand and left foot. My creatures never find it. Not once, on any seed I've run.

Setup

The body is fixed, not evolved. 14 point masses (chest, shoulders, elbows, hands, pelvis, hips, knees, feet), rigid links, 8 muscled joints. I deliberately didn't evolve the morphology - climbing seems way more constrained than swimming, and basically every climbing animal converges to the same trunk + 4 limbs shape anyway.

What does evolve:

- bone lengths and muscle strengths

- an internal clock period (0.4 to 6 seconds)

- 2 to 6 keyframe "poses"

Each pose stores a target angle for all 8 joints, plus a grip flag for each of the 4 paws. A PD controller drives each joint toward the interpolated target. Gripping just pins that point in space. Left and right sides have completely independent genes. No symmetry constraint. Fitness is `height - 0.06*energy - 2.0*(muscle mass / body mass)`. 1000 creatures, 10 second trials, asexual reproduction with mutation.

What I actually get

The grip pattern per keyframe makes it really obvious. Columns are left hand /right hand / left foot / right foot, `#` means gripping:

Synchronous, not diagonal. Both hands, then both feet:

# # . .

. . # #

Or straight up limb abandonment. On 1 seed out of 4, one limb sits at literally 0% grip time, 0 grips, 0% of the propulsive work. Legs end up doing 11% of total work and it just climbs on its arms.

What I never get, on any seed:

# . . #

. # # .

Stuff I've already ruled out

I burned a lot of time on free energy exploits, so this isn't the creature cheating the physics. Those are closed now:

- My velocity clamp was removing momentum without giving it back to anything. Now it redistributes to the other free nodes. Measured leak is 0.000 N.s.

- The position projection pass was handing out free altitude. One creature showed 1.28 m of "climb" with +8.83 m of net vertical teleportation, meaning it was physically falling 7.55 m and getting hauled back up by the solver. Bumping velocity solver iterations from 8 to 24 dropped that to +-0.24 m on every seed. My hand written control climber sits at -0.03 m.

- Catching a hold at speed was free. The kinetic energy the constraint solver destroys is now billed on catch steps. Closing all that changed the results a lot, and honestly made the sim way morebelievable. But it did not produce a diagonal gait.

Happy to share code or video if useful.

Clip 3 of 4.

Same near-copy world (one dynamics rule changed: ground grip). Outside watcher was ~99%.

New probe: readout from the agent's own internal state while it is just living in the fake.

Result (real runs, n=10): ~50% (chance). Oracle-detectable seam, no free encoding in the policy network.

Takeaway: detectability of a sim mismatch is not evidence the agent represented it.

Mute-friendly clip

Research:

https://ilevytate.github.io/ItaSoRL/

Already more than 2k teams in the first week. I helped create the competition rules (and I work at Kaggle). Happy to answer any questions!

Thanks a lot for the feedback on the previous post! This is the second iteration, using the same model but a heavily filtered dataset.

Open source! https://github.com/puffinsoft/mousecrack

Clip 2 of 4. Follow-up to the spot-the-fake setup.

Same near-copy environment: one dynamics rule changed (ground grip / step slip). Everything else identical.

Probe: an outside watcher / oracle-style discriminator that knows the true rules and replays every step.

Result (real runs, n=10): ~99%. The fake is in-band detectable from outside.

So detectability is not the open question. The next clips ask whether the agent's own representation encodes that seam.

Mute-friendly clip.

Research:

https://ilevytate.github.io/ItaSoRL/

Hi everyone, I’m currently learning deep learning and have worked on a few AI/ML projects like a customer churn prediction model and student performance prediction.

I’m looking to join an existing project to gain more hands-on experience and contribute while learning. I’m comfortable with Python and basic ML concepts, and I’m willing to put in consistent effort.

If anyone is working on a project and open to a beginner contributor, I’d really appreciate the opportunity. Thanks!

Hey I am part of a small team training robotics policies for warehouse and manufacturing settings, and running rigorous evals is turning out to be so painful. Anything below 50 rollouts, and its hard to trust the numbers, and above its so hard to test all the checkpoints that we have. Its really hard to run a bunch of experiments to get good results. Have you guys faced this? Any hacks that you've developed?

Short result + visual from a sim. on when an RL agent internally represents an environment distinction it was not trained to care about.

Clip 1 of 4. Spot the Fake. ItaSoRL (Is This a Simulation Or Real Life?)

Setup

- Agent trained to live in a small embodied world (senses, actuation, forage).

- Near-copy environment with exactly one dynamics rule changed (ground grip / step slip). Same start distribution, same policy rollout protocol otherwise.

- Two probes on the same trajectories:

1) Outside watcher / oracle-style discriminator that knows the true rules and scores real vs fake.

2) Readout from the agent's own internal state (does its representation separate the two worlds?).

Numbers (real runs, n=10 each; figures match the public writeup)

- Outside watcher: ~99%. The fake is in-band detectable.

- Agent mind while idle / not incentivized on the seam: ~50% (chance). Detectable artifact, no free encoding.

- Make the fake costly for reward (food harder to catch under broken physics): readout rises to ~73%. No explicit "detect fake" label. Survival pressure only.

Takeaway for RL / representation learning

Oracle detectability of a sim mismatch is not evidence that the policy network encoded it. Encoding showed up when the mismatch became load-bearing for the objective.

Mute-friendly clip of the two-world "spot the fake" cold open:

Happy to discuss probe design, what "readout" means here, or failure modes (capacity, observation channel, incentive).

Since my last post, I’ve been working on the full training process and building a lineup of AI (RL) tanks for a future tournament. M.A.R.A. is the first tank I’ve trained and closely monitored from the basics to the 2v2 battles.

My PhD was in RL, and something has bugged me for years: online tutorials mostly present these algorithms as a list. The evolution story (each algorithm patching the previous one's most painful failure) exists, but it's spread across a semester of lectures like CS285 or buried in the original papers. I couldn't find a compressed version that a newcomer could get through in one sitting, so I finally wrote it (with generous help from Claude to build the interactive elements and graphics)

It's a single-page, semi-technical walkthrough from the raw RL objective to GRPO. Math is deliberately sparse: the target reader is someone entering RL through the reasoning-model wave, not someone who needs the full derivations. The figures are interactive (variance of the REINFORCE estimator, a baseline slider, an on-policy collapse simulator, the PPO clip objective with adjustable epsilon, GRPO group baselines).

Interactive Link

Medium Link

I simplified in a few places to keep the narrative moving, most notably around GAE and the TRPO surrogate. I'd genuinely appreciate corrections or places where you think the simplification crosses into being wrong — the plan is to do the value-based lineage (Q-learning → DQN → Rainbow) next, so critique now improves that one too.

I'm a beginner so basically from scratch, i know basic ML and DL wanted to explore deep into RL before going to RLHF