Geography does matter, but it's not as critical as you may think it is.
College students who have not been to your church may not go to it if it's miles from campus. But many of them may be almost as unlikely to go to it if it's across the street from the student union.
While you certainly do want students to attend your church - and you may need to provide transportation if distance is an obstacle - many students will never go to your church unless it comes to them first. Your relationship with them needs to start on campus and/or at another location where they already go and are comfortable.
Meet with campus leaders to build positive relationships and discover at least one unmet need - as identified by the campus leader(s) - that your church can meet. If feasible, help current student members and participants form a registered student organization that's connected to your church so that the ministry has official access to the campus (e.g. reserve rooms for events).
I'll be glad to help you find open doors on campus. Send me an email at email@example.com for more information.