But it looks to me like it’s wear.
Some of the finish has clearly been lost here (the grey areas) and it's very likely that it is partly due to wear given the location, but weathering effects are sure to be part of the underlying cause. The original film finish or semi-penetrating finish has been damaged by UV and water and this will start the process of it flaking/peeling from the surface even without direct wear.
And I don’t know if I should sand it.
In refinishing it's often ideal to get back to bare wood and begin again, but often outdoor pieces don't have to look quite as perfect as interior furniture so this isn't standard practice and top-up work is more commonly done. This generally involves cleaning, light surface sanding and the application of a fresh coat or two of finish. In this case I think it's quite possible you'll be satisfied with the results achieved by applying a fresh coat of finish after a good scrub and a light overall sanding (with more and heavier sanding of upper areas, esp. horizontal surfaces).
This is a case-by-case thing as you'd expect, but from the background of the photo it looks like a lot of the finish is in good shape and just needs a refresh to continue looking good and providing protection.
If you power-wash the pieces this could considerably lessen the amount of sanding you need to do. It could lift off quite a bit or all of the remaining finish that's been damaged by weathering. See the images in this previous Answer for the quite remarkable transformations possible with power washing.
After this or any other washing/cleaning procedure let the pieces thoroughly dry for 1-2 days. Then sand every surface as outlined above, remove the dust1 and apply fresh finish.
Some experimentation may be required to find a suitable finish, but a product like commercial "Teak oil" may be a good match for what's on these already.
Note: this process of lightly sanding the surface and re-applying finish is the maintenance schedule for finished patio furniture like this. It needs to be done regularly2, forever. You can't skip or delay a scheduled refinish without consequences, and rectifying the issues will entail much more work.
The alternative, much more acceptable to many owners, is to remove as much finish as effort allows to get as close to bare wood as possible and then deliberately leave it to weather to grey. Not only does this greatly lessen the maintenance work required (you only have to clean once or twice a year) but many find the grey colour very attractive.
1 It's advisable to wear a good dust mask during sanding and dust removal, even if working outdoors. Tropical woods like teak are known irritants and sensitisers.
2 Yearly if the conditions are challenging (near the sea, lots of sun and/or rain), every 2-4 years for milder locations, 3-5 years where there's some cover.