I think is better to not use an standard port and using fail2ban at the same time to avoid automated attacks. If you manage to implent what you are looking for, you are potentially telling an stacker which accounts exist and which not, allowing him to do an easier brute force attack. A typical attacker using a botnet will not be stopped by a single IP being baned, and as son as an IP is banned he will know that this account doesn’t exists. Another option is enabling port knocking.
If I’m bruteforcing a server and each time that I try an username/password my IP gets banned but suddenly one combination allows me to do 4-5 test ( any bigger number than previously) you are potentially telling me that this user is different (it exists) than the previous ones. Therefore you are doing the attack easier for me because now I know which users actually exist in the machine. It doesn’t matter if you are locking the attacker after the password was given.
As others told you, using public key auth, non standard ports or even port knocking will be much more useful.