I found a way to open the file explorer, but it was nested inside the new window I created for it. Is there a way where I can just open the file explorer window itself?
You'll need to use the FileChooser object, which will open your OS's File Explorer. Take a look at the Movie Collection repository on GitHub for reference.
Use the FileChooser object from the javafx.stage library
As people have been saying FileChooser is amazing or Java Swing I believe has JFileChooser.