I would like to let EAC automatically create directories named from artist or album name. Is this possible somehow?

You can set this in the filename option. If you use the ‘\’ character, EAC will create all these subdirectories. For example %a\%n – %t
But you may not specify an absolute path like c:\%a\%n nor \%a\%t
For more information read the tooltip of this option.