I just replaced both upstream and downstream O2 sensors. Should the engine light go off right away?

It's possible that the engine light will go off right away, but it's also possible that it may take a little bit of time. If the light doesn't go off immediately, give it a day or two to see if it resets itself. If the light doesn't go off after a few days, you may need to take it in for a diagnostic test to see the problem.