I went to Neversink Pit recently. It is worth the effort of a) finding it and b) hiking essentially up to it. The location is obscure, the parking lot is an easily missed gravel road on your right (from the only direction you would approach it from). Parking is limited, you must visit the website and obtain a permit online, it is free. The trail is obscure, at least in the fall with the leaves down but there are occasional trail markers. It takes about 20 to 30 minutes to hike up to from the parking lot. It is somewhat strenuous as it is mostly uphill at a 20 to 25 degree angle but it is worth it. There are no personnel, no facilities, probably no other visitors either. I shall return, it is beautiful and worth going out of your way to.
Is this what you are writing about? If so, it sounds very beautiful, indeed. Thank you for alerting me to the existence of this place.
Your cave link is the place. Obscure and a bit of a hike but worth it.