Initially you’re kind of guessing. You’re basically estimating to the best of your ability and based on what you discuss and agree on with your Agile Team. You’re also taking into account the number of people in the Agile Team (so how many people are executing user stories), their complexity (based on the story points) and also your gut feeling. After some Sprints have gone by, you will be using your Velocity as a way of defining how many story points you will add to a Sprint.
Assigning user stories to a Sprint without assigning story points first is not ideal nor recommended. But in practice, it could occur and Agile teams would always remain flexible. So they would assign user stories to the best of their ability using other factors like the ones mentioned above. Remember, keep it simple. Agile is about simplicity. It’s also about being flexible, moving fwd quickly, making decisions quickly and not wasting too much time over thinking things which are not really that important. Plus, keep in mind that in Agile you’re constantly reflecting on past performance and making improvements on the fly (based on your Retrospectives). So even if you made a mistake in your assumptions when assigning user stories to a Sprint, it would become evident to you and the team pretty quickly and you would adjust and iterate, the Agile way!