So this is a little late, but the National First Ladies Library Blog did a nice 5 part piece on the history of the Easter Egg roll at the White House.
Check out this blurb from Part 2:
Documentation does show, however, that a far more obscure First Lady was likely the first to appear at a White House Easter Egg Roll event.
In his memoirs, White House clerk William Crook recorded the fact that, despite her chronic condition of tuberculosis limiting most of her public appearances on the state floor at public social events, First Lady Eliza Johnson came out onto the South Portico to watch her five little grandchildren rolling colored eggs on Easter Monday and taking great delight in watching their games.
There was no mention, however, about whether there were other children present.
What is established as fact is that, formally or informally, sometime before or after the Civil War era, children of Washington were coming to the greensward of the U.S. Capitol Building the Monday after Easter Sunday, where its sweeping lawn provided the perfect place to hold contests to see who could roll their brightly-colored dyed Easter eggs along with a spoon the fastest.