What I can tell you is the country around it. Culfa, the district town, lies about five kilometres away.
Nakhchivan is a place of bare mountains, red and ochre in the sun, with irrigated green strips along the river, apricot and pomegranate orchards, and ancient fortresses on the crags. Alinja Castle, near Julfa, is a medieval hilltop fortress on a steep rock, with views over the valley. Local tradition places Noah's landing here after the flood, and the name of the main city is sometimes linked to that story. The climate is continental, with hot dry summers and cold winters, and the sky is usually clear.
Riding in this part of the world requires planning. The roads along the Aras and into the mountains are generally paved but narrow, and the roads climb in bends through arid hills with long views, but surfaces can be patchy and livestock wander freely. Frontier zones near the Iranian border have checkpoints and restrictions, so carry passports and documents and do not wander near the border without permission. Travel into Nakhchivan itself involves flying or crossing through Iran or Turkey, so check visa and vehicle rules in advance.
Practically, Culfa has fuel and basic shops, and Yaycı and Sumbatan-diza are smaller villages with little. In the towns, restaurants serve kebabs, lamb stews, bread baked in clay ovens and strong tea in small glasses. Bring water and sun protection, and get an early start, since midday heat on the bare hills is severe. If you find the chapel, enjoy the silence, and treat it with respect as a place of worship.