Planning a back lit razor measurement of a super tele lens

[Triforian]

Hi Frans,

I'm trying to evaluate sharpness of a recently bought Fujinon 150-600 on my X-S10 body. Fujifilm show very good performance at 45 lp/mm on the wide and tele end. Optical limits found an mtf50 of >3000 lw/ph near center 150-600mm wide open. Digital Camera World found >1500 lp/ph (? no axis label, smh) 150-400mm. So the reviews seem to agree. And from what I calculated, this would correspond to 0.33 c/p on my 4160 pixels high sensor, correct?

I played around with mtf mapper and an A4 printed lens grid on the weekend. Long story short: the best I could get from there lens was an average of 0.18 c/p around the center at 150mm. Since I'm only interested in the center I put the chart far enough away that I get about 8-10 printer dots per pixel width, about equivalent to an A0 chart. I used auto focus after confirming that it consistently yielded better results than my manual focus. I also used electronic shutter and deactivated image stabilization to exclude other sources for motion blur. I don't have professional lights, so lighting isn't very even, but CNR looked good in all edges.

Just to make sure, I tested my Sigma 56mm dc dn at f/4 with the same method and identical chart and found about 0.33 c/p concluding that my setup should be good enough to show if the fujinon is as sharp as claimed.

I'll make a last attempt with a back lit razor. I'm assuming I should use the "single edge" option in mtf mapper for that. Do you think it's reasonable for me to expect being able to measure the Fujinon to the accuracy I'm aiming for? Any additional hints?

I attached center crops showing the chart shots I mentioned.

[Frans van den Bergh]

Hi Florian,

I think your calculation is in the right ballpark. So keeping in mind that one cycle = one line pair, 1500 lp/ph -> 1500/4160 = 0.3606 lp/ph.

Looking at your attached sample crops, I see that your edge lengths are trending to the lower end of the range. Not a problem in itself, but if you are capturing an A4 chart from a large distance it might be (slightly) better to try and print the central A4 part of an A3 chart (on A4 paper) so that the trapezoidal shapes are a bit larger, giving you greater edge lengths. This should reduce variability a bit, but your crops are still good enough as-is.

I don't have any direct experience with on-sensor PDAF sensors, so I do not know the recipe for getting the best results out of them, but keep in mind that DSLR-style PDAF does not usually give me the highest possible MTF50 values. Lenses designed for autofocus might not have enough of a focus throw to really do much better than the AF system, so the most practical way to determine if you have optimal focus is to use some kind of focusing rail. Alternatively you can just capture a lot of through-focus shots, i.e., a sequence that starts out front-focused, and ends with back-focused images, and hope that one of them is close to the optimal focus distance. The reason I emphasise optimal focus here is that this is the most common challenge in measuring a system's peak MTF values. At least, I find it pretty hard to hit optimal focus ... :)

It sounds like you are also aware of the additional challenges when capturing sharp images at longer focal lengths (shutter shock, etc), but it is hard to completely rule out those factors unless you have an experiment that systematically investigates, for example, MTF50 vs shutter speed. So there could still be some technique-related factors that are causing your lower-than-expected results at 150 mm.

For the back-lit razor experiments you will likely find focusing even harder (again, through-focus sequences are your best bet). Rather than the single-edge MTF Mapper option, I would recommend that you use "File/Open with manual edge selection"; this gives you better control over which part of the razor blade edge you want to use for the measurement (I found that normal double-sided razor blades have a slight curvature near the ends of the blade, so I tend to select only the central part of the blade edge).
But given that you are getting > 0.32 cycles/pixel with the 56 mm lens, I would say that your A4 charts have enough resolution for the 150 mm lens to produce comparable results, but you are of course welcome to try a razor blade too :)

Lastly, do you know if you are using the same raw development settings as the review sites you are comparing to? Are your Sigma 56 mm results comparable?

So if I had to summarise, I would say you are most likely not reaching optimal focus with the 150 mm lens, or some raw development software differences between your set-up and the review sites could be at play (unfortunately MTF Mapper does not support raw processing on x-trans sensors).

Regards,
Frans

[Triforian]

Hi Frans,

thank you very much for your advice. I had underestimated the impact of focusing. I tried to solve that by using the focus bracket function on my camera body. I also wasn't aware that mtf mapper doesn't do raw processing of x-trans files. I can imagine it would be a lot of extra work. Therefore I decided to my RAW processing in Capture One which I am familiar with.

Unfortunately I had no references of how the reviewers did their processing. They only state that they use the Imatest software. Therefore I decided to crop, increase the exposure by +3 stops and add +15 contrast (whatever that means in acutal units, I'm a bit unsure). I determiend these settings using the highlight / shadow warning to make sure none of the pixels were overexposed and most weren't underexposed (the setting in software usese values of 1 to 254 in a span of 0 to 255, but the RAW works with more than 8 bits to my understanding).

I exportet into grayscale 16 bit tiff with a gamma of 1.0 and used the single edge option in mtf mapper. My reference Sigma 56mm now reached 0.43 c/p at optimum focus, i.e. ~110 lp/mm . That surpasses what I had expected based on a Lenstip review which claimed 80 lp/mm. However the site states that they use autofocus in three passes, so they might have missed optimal focus. I can also imagine that my processing pushed the results above the actual value due to the contrast increase. The LSF looked a bit more triangular than I would have expected.

Regardless, even with these settings the 150-600 at 200 only reached 0.23 c/p, falling slightly short of my requirement of 0.24 c/p. I also tried to reference vs. a stopped down 70-300 which should reach similar values and got 0.29 c/p at 200 mm. I was therefore convinced that I wasn't limited by my method to values lower then the requirement.

I conducted a final measurement at 600 mm, which is the most important part of this lens for me and found 0.16 c/p. I know that there are a lot of uncertainties associated with this value. Unfortunately I had to make a judgement call at that point and decided to use the 14-day no questions asked return policy of the vendor. I of course informed them of my findings, but can only hope that they have the decency to test it or better send it back to Fujifilm for testing and maybe repair.

If I hadn't been under time pressure since the deadline for claiming the return runs out today, I think I would have enjoyed this very much. To a large part because of mtf mapper and your advice. Thank you!

Cheers,
Florian