To open the Colosseum you will first need to complete the quest "Where's the Receptionist?" on Planet Roak in the Astralian Continent, the city of Tatroi. After doing so, you will then need to fight many battles to progress up the rankings and you can only challenge someone three spots ahead of you to move up. You may need to be around level 110 to get the top ranking for the team boards, but the solo can be done sooner. Solo matches are best won using Edge or Meracle, while team matches you should use Bacchus and his Black Hole Sphere to deal with multiple enemies at once, or just fight however you are most comfortable with your other fighters (I personally use Edge, Meracle, Sarah and either Myuria or Reimi). You should work on the team ladder since you will need the #1 ranked person for your Monster achievement and a few Battle Trophies.
Note: The option to fight for the top spot of each ladder won't show up until you fight the #2 fighter a few times in a row. Just keep killing them until #1 can be selected.