The Rust-colored Egg is one of two eggs found near the end of Silent Hill 2. James Sunderland obtains the egg after defeating the Pyramid Head duo in the first floor Lobby of the Otherworld Lakeview Hotel, after coming to terms with his guilt.
The connection the eggs have to Pyramid Head and James isn't specified in-game, but they are both used to unlock the doors leading to the final boss. Once both are inserted into the lobby doors, they will unlock and grant access to the final hallway.
The rust-colored egg could symbolize Mary Shepherd-Sunderland, as she could be represented as "old", "diseased", "ugly", "dead", while the scarlet egg could represent Maria, who is "new", "healthy", "beautiful", "alive".
The rust-colored egg could also symbolize Mary as "old blood", while the scarlet egg could symbolize Eddie Dombrowski as "new blood", and the regret and guilt James feels for each murder, although they were both for different reasons. In addition, it has been confirmed that the two Pyramid Heads represent Maria and Mary, so this theory is unlikely.