If the first place you go to is full, don’t panic. First off, ask the owner/staff of the place that’s full if they know of any place that isn’t; you can be sure that he knows his local competition. Also ask if you can call from there, and potentially save yourself a long walk. If the owner can’t help you at all, or if everything is full, return to the train station and start the whole process over. (This is when you will be blessing your light back-packs, and swearing to call me and thank me for harping on the idea of going light. You will forget as soon as you find a bed, but that’s only human.) You may have to adjust the maximum amount you’re willing to pay if things get really bad.

In a large city, one option to consider before paying a lot more is “commuting”: staying outside of that city and traveling in by train each day. The accommodations you find in a suburb or small town will probably be cheaper, nicer, and less crowded than in the city center, and the money you save will probably offset the cost of the commute, especially if you have an unlimited rail pass that you can use for the daily round trip. If you’re hosteling, you may find yourself doing this anyway, as hostels are often situated outside of city centers.
In a city or a small town, if it really looks as if you may not find any accommodation at all, and you can’t commute, before you spend another five hours looking, give yourself an out. Check the train schedules. Find one going to a place listed in your guidebook. Call ahead to make a reservation. At least that way there is a bed somewhere with your name on it. If your search for a bed does come up empty, you can always get on a train and go to your reservation. And if all else fails, you can always catch a night train, with or without couchette.

Please be reassured that situations like this are rare, except in cases of late arrival in major cities or tourist centers, or if there is a major event taking place in the area (such as Easter in Nice, Oktoberfest in Munich, etc). In almost every case spending more money will guarantee finding a room. Rooms in the nicer hotels in a town will still be available long after all hostel and budget rooms in a city have sold out. If it comes to this, call around to the local hotels first. Don’t expect a discount if you go in person; once you’re in the lobby, the hotel knows they’ve got you.