Yes, for some taxes. However, if you complete Form EFT-1, Authorization Agreement for Electronic Funds Transfer, then you will have the option to pay by EFT.