Parshas Miketz III: Yehuda's Cheshbon

At the end of the parsha, Yosef wants to imprison Binyamin and let the other brothers go free. Yehuda steps up and says that Yosef should imprison everyone including Binyamin. Only when Yosef refuses and says that he will only take Binyamin, does Yehuda change his position and prepare for battle (vayigash elav Yehuda). The meforshim all explain that originally Yehuda thought they were being punished for having sold Yosef. If that is the case then there is nothing to do but except their fate. However, when he saw that only Binyamin was going to be punished, he realized that it had nothing to do with mechiras Yosef, and therefore he prepared for battle.
The question that remains is that if Yehuda originally thought that they were being punished for selling Yosef, then why did he agree to let Binyamin be taken as a slave as well? Binyamin was not involved in the mechira so why should he be punished.
L'chora you could answer based on a Sefer HaIkkarim. The Sefer HaIkkarim discusses why a tzaddik woud be punished in this world. One possibility he mentions is that when there is a gezairah on the tzibbur, then even the individual tzaddikim are not spared. It could be that this was Yehuda's cheshbon as well. Granted Binyamin was innocent but once HKBH made a gezairah on the 10 brothers, so even Binyamin had to be punished.
I subsequently saw this idea in the Netziv, The Emek Davar (44:16) says that Yehuda could not be moser nefesh to save Binyamin, because once this was ratzon hashem there was no hope.