The Rust-colored Egg is one of two eggs found near the end of Silent Hill 2. James Sunderland obtains the egg after defeating the Pyramid Head duo in the first floor lobby of the Otherworld Lakeview Hotel after coming to terms with his guilt.
The connection the eggs have to Pyramid Head and James isn't specified in-game, but they are both used to unlock the doors leading to the final boss. Once both are inserted into the lobby doors, they will unlock and grant access to the final hallway.
The rust-colored egg could symbolize Mary Shepherd-Sunderland, as she could be represented as "old", "diseased", "ugly", and "dead", while the scarlet egg could represent Maria, who is "new", "healthy", "beautiful", and "alive".