Those are basic features om most handheld units. Most also have a screen showing a pointer indicating the direction and distance to the point. Coordinates are commonly stored in 'waypoints' which can be navigated to using the pointer or other screens including a plot showing the waypoint and your present position.
The Garmin eTrexH is a good basic model (link)
However it only has a serial interface and does not come with a computer cable. The eTrex Venture® HC (link)
comes with USB interface and cable and has a color display as well.