Can it be considered as a כרמלית, etc. - מהו שתעשה ככרמלית
A hand extended in another רשות is generally not considered a כרמלית. The only possibility that it may be considered a כרמלית is whether he may return an extended laden hand to the רשות where his body is.
Did the רבנן fine him not to return it - מי קנסוה רבנן לאהדורה
The query of אביי was whether the רבנן enacted an איסור כרמלית to prohibit the person from retracting his hand, but not that they (merely) punished him and prohibited him to retract his hand. The prohibition of איסור כרמלית (even if it based on a קנס) extends even to cases of בשוגג מבעו"י as opposed to a mere קנס which will prohibit him only במזיד ומשחשכה.
And if you want I can say; really you cannot resolve the query - ואיבעית אימא לעולם לא תפשוט
The ר"י maintains that the query of רב ביבי is resolved. However, the רשב"א maintains that it is not resolved, and even initially the גמרא meant that we can resolve either the query of אביי or of רב ביבי but not either one specifically.
They did not fine him by שוגג, etc. - בשוגג לא קנסוה
The current answer of כאן בשוגג is a case of מבעו"י and it cannot resolve the בעיא דרבב"א. If we are forced to assume that משחשכה קנסוה [only] because מבעו"י קנסוה, then it does not prove that לא התירו לרדותה.