It can be.

If you’re considering having sex for the first time, this probably looms like more of a potential problem than it actually is.

Hopefully you’ve addressed the more important issues along with this, such as …

1. Having (a) respectful, considerate, patient, non-abusive partner(s).
2. Making sure you’ve discussed protection against STIs and/or unwanted pregnancy, knowing to a very accurate degree how to protect against them, and having unimpeded access to that protection.
3. Having a place where you’ll be uninterrupted and unhurried.
4. Understanding that consent is ongoing, situational, and can be withdrawn by any partner at any time for any reason.
5. Deciding you want to do it because it’s what you and your partner(s) truly want to do, not (a) to hold onto a partner that threatens to leave you if you don’t do what they want, (b) as a way to protect against loneliness, (c) because you think everyone else is doing it and are being pressured by your colleagues to “join the club,” or (d) because you think you’ll be a different, “better” person after having had sex.
6. Understanding that sex is not just PIV, PIA (anal), and fellatio.

There are a bunch of other situational concerns you could address that I believe take precedence over the fact that the first times might be awkward, but let’s actually address that awkwardness.

Congratulations on admitting that you haven’t had partnered sex. Lots of other people pretend to have had sex in order to impress others. You are head and shoulders above those people because the truth actually matters to you. Ethical, consent-respecting partners will appreciate this. This honesty will make you a great sex partner because if something isn’t working, you’ll feel free and comfortable informing your partners of this. Conversely, if something is working, you’ll feel free and comfortable letting them know this as well.

Honesty, a willingness to speak up for yourself, asking questions, and allowing yourself to be pleased and please will help you to have better and better sex as you gain experience. Good, respectful partners love and expect these qualities in others.

Kudos for demonstrating these qualities with your question. Your future partner(s) should be highly impressed by your integrity, not any sort of performance you might think you’re supposed to demonstrate the first few times.