You probably already have this resolved, but this is my area of expertise! You have a couple of solutions here.
1.) Wire via Ethernet. Pro, direct connection=better speed. Con=Long wire hanging around.
2.) Second Wireless Modem. Pro, will provide a better signal locally to you. Con, connections between the two modems can still be spotty for a number of reasons from building material and intervening objects (especially metal ones, massive amounts of electronics and electrical. )
3.) High Powered Router. Pro, a single router has a much large signal distance. You usually want to add an larger external antenna to these to maximize the distance. Con, can be comparatively expensive for a decent brand such as CradlePoint, can still be affected by large amounts of metal and electrical, can be harder to setup for the non-technical.
One troubleshooting measure you can try is to just change the placement of the modem and it's antennas. The signal off the antenna is a rough donut shape, with the Omni directional shape being the most common
. Since you're on the second floor I'm pretty sure that you're only touching the edge of this wireless signal. If there is an adjustable antenna, point it away from vertical about 45 degrees and try again. Or try moving the entire router a few feet.