[BUG-FIX] Mail quota not working LMTP & LDA (sieve)

drMacFaulty · Post by **drMacFaulty** » Fri Feb 20, 2015 9:07 pm

edit: Please see post below for solution to get quota working with LDA and/or LMTP transports and for a working LDA configuration (imho the config still needs some polishing). I thought to post my 'semi-finished' config files here, as I have experienced that the documentation of Exim and Dovecot is quite hard to get through.

Hello,

After configuring dovecot to use lmtp with exim and enabling antispam module, quota was still working with current accounts. If I create a new mailbox, then the quota amount is also visible in the passwd files (/etc/exim4...). However, if I run doveadm quota get -u <e-mail address> then there is no limit set on STORAGE (the value is present in the passwd files, though) as you can see below. If I adjust the quota of an already existed account or new account (doesn't matter), then quota doesn't get set or stays at the old value.

Could it be the issue that the v-add-mail-account script uses doveadm to create a new account? Should it perhaps use lmtp to create a new account (although this doesn't make any sense to me).

Code: Select all

$ doveadm quota get -u <e-mail address>
Quota name Type    Value Limit     %
           STORAGE 21926     -     0
           MESSAGE     2     -     0

Output of passwd file of the domain results in the following data:

Code: Select all

$ cat /etc/exim4/domains/<user-domain>/passwd 
test1:<pass-hash>:<username>:mail::/home/<username>:100
test2:<pass-hash>:<username>:mail::/home/<username>:50

As you can see, quota gets set, but gets lost somehow / somewhere.

Can anybody help me to solve this annoying issue? IMAP and all works beautifully, but only quota doesn't get through anymore. Quota is visible in the control panel, though.

My dovecot config:

Code: Select all

$ doveconf -n
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-042stab092.2 i686 Debian 7.8 
auth_mechanisms = login plain
auth_verbose = yes
base_dir = /var/run/dovecot/
log_path = /var/log/dovecot/dovecot.log
mail_access_groups = mail
mail_location = maildir:%h/mail/%d/%n
mail_plugins = " quota"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
namespace inbox {
  inbox = yes
  location = 
  mailbox Archive {
    auto = subscribe
    special_use = \Archive
  }
  mailbox Archives {
    auto = no
    special_use = \Archive
  }
  mailbox "Deleted Messages" {
    auto = no
    special_use = \Trash
  }
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    auto = no
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix = 
}
passdb {
  args = scheme=MD5-CRYPT username_format=%n /etc/exim4/domains/%d/passwd
  driver = passwd-file
}
plugin {
  antispam_backend = pipe
  antispam_pipe_program = /usr/bin/sa-learn
  antispam_pipe_program_notspam_arg = --ham
  antispam_pipe_program_spam_arg = --spam
  antispam_spam = Junk
  antispam_trash = Trash
  quota = maildir
  quota_rule = Trash:ignore
  quota_rule2 = Spam:ignore
  sieve = ~/.sieve/%d/%n/.dovecot.sieve
  sieve_before = /var/lib/dovecot/sieve/default.sieve
  sieve_default = /var/lib/dovecot/sieve/default.sieve
  sieve_dir = ~/.sieve/%d/%n/scripts
  sieve_global_dir = /var/lib/dovecot/sieve/
  sieve_max_script_size = 1M
}
protocols = lmtp sieve imap pop3
service auth {
  unix_listener auth-client {
    group = mail
    mode = 0660
    user = dovecot
  }
  user = dovecot
}
service lmtp {
  executable = lmtp -L
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  process_min_avail = 1
  service_count = 1
  vsz_limit = 64 M
}
service managesieve {
  process_limit = 512
}
ssl = required
ssl_cert = </usr/local/vesta/ssl/certificate.crt
ssl_key = </usr/local/vesta/ssl/certificate.key
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = username_format=%n /etc/exim4/domains/%d/passwd
  driver = passwd-file
}
protocol imap {
  imap_idle_notify_interval = 2 mins
  mail_plugins = " quota imap_quota antispam"
}
protocol lmtp {
  info_log_path = /var/log/dovecot/dovecot-lmtp.log
  log_path = /var/log/dovecot/dovecot-lmtp-errors.log
  mail_plugins = " quota sieve"
}

drMacFaulty · Post by **drMacFaulty** » Wed Feb 25, 2015 7:30 pm

It is fixed! Works on a Debian Wheezy box with Exim 4.80 and Dovecot 2.1.7. Can this be added as a patch in these files within the next release for users running dovecot-lmtp or dovecot-lda (a check for this by polling dovecot configuration)?

Patch is needed in the following vesta executables:
- v-add-mail-account
- v-change-mail-account-quota

Variables added in files:

Code: Select all

# Argument defenition
...
# "K" = Kbytes, "M" = Mbytes, "G" = Gbytes ("M" is used in vestaCP).
multiplier="M"

v-add-mail-account changes:

Code: Select all

Original:
str="$account:$md5:$user:mail::$HOMEDIR/$user:$quota"

Patch:
str="$account:$md5:$user:mail::$HOMEDIR/$user::userdb_quota_rule=*:storage=$quota$multiplier"

v-change-mail-account-quota changes:

Code: Select all

Original:
str="$account:$md5:$user:mail::$HOMEDIR/$user:$quota"

Patch:
str="$account:$md5:$user:mail::$HOMEDIR/$user::userdb_quota_rule=*:storage=$quota$multiplier"

Sources (for reference and on which I've partially based/adjusted the config files on):
http://wiki2.dovecot.org/LDA
http://wiki2.dovecot.org/LDA/Exim
http://wiki2.dovecot.org/LMTP
http://wiki2.dovecot.org/LMTP/Exim
http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples
http://www.exim.org/exim-html-current/d ... sport.html
http://www.exim.org/exim-html-current/d ... ports.html
http://dokuwiki.tachtler.net/doku.php?i ... cot_quotas (helped me in discovering the wrong part in the passwd files).

---

Please excuse me for the bold parts, but there are always some users who do not read everything.

NOTE: This config is tested on Debian Wheezy with exim4 and dovecot 2.x. I cannot guarantee this will work for you, so use at your own risk!!
NOTE: For this config to work, there needs to be a dovelda user who can ONLY EXECUTE dovecot-lda AS ROOT (by using sudo)! I found this kinda less insecure than running LMTP (which is constantly under root), according to dovecot 2.x docs.

The configuration of dovecot (doveconf -n dump), change to your needs:

Code: Select all

# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-042stab092.2 x86_64 Debian 7.8 
auth_mechanisms = login plain
auth_verbose = yes
base_dir = /var/run/dovecot/
log_path = /var/log/dovecot/dovecot.log
mail_access_groups = mail
mail_debug = yes
mail_location = maildir:%h/mail/%d/%n
mail_plugins = " quota"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
namespace inbox {
  inbox = yes
  location = 
  mailbox Archive {
    auto = subscribe
    special_use = \Archive
  }
  mailbox Archives {
    auto = no
    special_use = \Archive
  }
  mailbox "Deleted Messages" {
    auto = no
    special_use = \Trash
  }
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    auto = no
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix = 
}
passdb {
  args = scheme=MD5-CRYPT username_format=%n /etc/exim4/domains/%d/passwd
  driver = passwd-file
}
plugin {
  antispam_backend = pipe
  antispam_pipe_program = /usr/bin/sa-learn
  antispam_pipe_program_notspam_arg = --ham
  antispam_pipe_program_spam_arg = --spam
  antispam_spam = Junk
  antispam_trash = Trash
  quota = maildir:User quota
  quota_rule = *:storage=10M
  quota_rule2 = Trash:ignore
  quota_rule3 = Spam:ignore
  sieve = ~/.sieve/%d/%n/.dovecot.sieve
  sieve_before = /var/lib/dovecot/sieve/default.sieve
  sieve_default = /var/lib/dovecot/sieve/default.sieve
  sieve_dir = ~/.sieve/%d/%n/scripts
  sieve_global_dir = /var/lib/dovecot/sieve/
  sieve_max_script_size = 1M
}
protocols = lmtp sieve imap pop3
service auth {
  unix_listener auth-client {
    group = mail
    mode = 0660
    user = dovecot
  }
  unix_listener auth-userdb {
    group = mail
    mode = 0660
    user = dovelda
  }
  user = dovecot
}
service lmtp {
  executable = lmtp -L
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  process_min_avail = 1
  service_count = 1
  vsz_limit = 64 M
}
service managesieve {
  process_limit = 512
}
ssl = required
ssl_cert = </path/to/ssl.crt
ssl_key = </path/to/ssl.key
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = username_format=%n /etc/exim4/domains/%d/passwd
  driver = passwd-file
}
protocol imap {
  imap_idle_notify_interval = 2 mins
  mail_plugins = " quota imap_quota antispam"
}
protocol lda {
  auth_socket_path = /var/run/dovecot/auth-userdb
  info_log_path = /var/log/dovecot/dovecot-lda.log
  log_path = /var/log/dovecot/dovecot-lda-errors.log
  mail_plugins = " quota sieve"
  postmaster_address = [email protected]
}
protocol lmtp {
  info_log_path = /var/log/dovecot/dovecot-lmtp.log
  log_path = /var/log/dovecot/dovecot-lmtp-errors.log
  mail_plugins = " sieve quota"
}
protocol pop3 {
  mail_plugins = " quota"
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}

Exim4 configuration (exim4.conf.template, combined configuration file), change to your needs:

Code: Select all

##########################################################################
SPAMASSASSIN = yes
SPAM_SCORE = 50
CLAMD =  yes
##########################################################################

domainlist local_domains = dsearch;/etc/exim4/domains/
domainlist relay_to_domains = dsearch;/etc/exim4/domains/
hostlist   relay_from_hosts = 127.0.0.1
hostlist   whitelist = net-iplsearch;/etc/exim4/white-blocks.conf
hostlist   spammers = net-iplsearch;/etc/exim4/spam-blocks.conf
no_local_from_check
untrusted_set_sender = *
acl_smtp_connect = acl_check_spammers
acl_smtp_mail = acl_check_mail
acl_smtp_rcpt = acl_check_rcpt
acl_smtp_data = acl_check_data
acl_smtp_mime = acl_check_mime

.ifdef SPAMASSASSIN
spamd_address = 127.0.0.1 783
.endif

.ifdef CLAMD
av_scanner = clamd: /var/run/clamav/clamd.ctl
.endif

#openssl_options = +no_sslv2 +no_sslv3
tls_require_ciphers = NORMAL:!VERS-SSL3.0
tls_advertise_hosts = *
tls_certificate = /path/to/ssl.crt
tls_privatekey = /path/to/ssl.key

daemon_smtp_ports = 25 : 465 : 587 : 2525
tls_on_connect_ports = 465
never_users = root
host_lookup = *
rfc1413_hosts = *
rfc1413_query_timeout = 5s
ignore_bounce_errors_after = 2d
timeout_frozen_after = 7d

DKIM_DOMAIN = ${lc:${domain:$h_from:}}
DKIM_FILE = /etc/exim4/domains/${lc:${domain:$h_from:}}/dkim.pem
DKIM_PRIVATE_KEY = ${if exists{DKIM_FILE}{DKIM_FILE}{0}}


##########################################################################
begin acl
##########################################################################
acl_check_spammers:
  accept hosts = +whitelist
  drop    message       = Your host in blacklist on this server.
          log_message   = Host in blacklist
          hosts         = +spammers
  deny    message       = rejected because $sender_host_address is in a black list at $dnslist_domain\\n$dnslist_text
          dnslists      = ${readfile {/etc/exim4/dnsbl.conf}{:}}
  accept

acl_check_mail:
  deny
          condition     = ${if eq{$sender_helo_name}{}}
          message       = HELO required before MAIL
  drop
          condition     = ${if isip{$sender_helo_name}}
          message       = Access denied - Invalid HELO name (See RFC2821 4.1.3)
  drop    message       = Helo name contains a ip address (HELO was $sender_helo_name) and not is valid
          condition     = ${if match{$sender_helo_name}{\N((\d{1,3}[.-]\d{1,3}[.-]\d{1,3}[.-]\d{1,3})|([0-9a-f]{8})|([0-9A-F]{8}))\N}{yes}{no}}
          condition     = ${if match {${lookup dnsdb{>: defer_never,ptr=$sender_host_address}}\}{$sender_helo_name}{no}{yes}}
          delay         = 45s
#  drop
#          condition     = ${if match{$sender_helo_name}{\N^\[\N}{no}{yes}}
#          condition     = ${if match{$sender_helo_name}{\N\.\N}{no}{yes}}
#          message       = Access denied - Invalid HELO name (See RFC2821 4.1.1.1)
  drop
          condition     = ${if isip{$sender_helo_name}}
          message       = Access denied - Invalid HELO name (See RFC2821 4.1.3)
#  drop
#          condition     = ${if match{$sender_helo_name}{\N\.$\N}}
#          message       = Access denied - Invalid HELO name (See RFC2821 4.1.1.1)
#  drop    message       = "REJECTED - Bad HELO - Host impersonating [$sender_helo_name]"
#          condition     = ${if match{$sender_helo_name}{$primary_hostname}}
  drop    condition     = ${if eq{[$interface_address]}{$sender_helo_name}}
          message       = $interface_address is _my_ address
  accept


acl_check_rcpt:
  accept  hosts         = :

  deny    message       = Restricted characters in address
          domains       = +local_domains
          local_parts   = ^[.] : ^.*[@%!/|]

  deny    message       = Restricted characters in address
          domains       = !+local_domains
          local_parts   = ^[./|] : ^.*[@%!] : ^.*/\\.\\./

  require verify        = sender

  accept  hosts         = +relay_from_hosts
          control       = submission

  accept  authenticated = *
          control       = submission/domain=

  require message       = relay not permitted
          domains       = +local_domains : +relay_to_domains

  deny    message       = smtp auth requried
         sender_domains = +local_domains
         !authenticated = *

  require verify        = recipient

.ifdef CLAMD
  warn    set acl_m0    = no
  warn    condition     = ${if exists {/etc/exim4/domains/$domain/antivirus}{yes}{no}}
          set acl_m0    = yes
.endif
.ifdef SPAMASSASSIN
  warn    set acl_m1    = no
  warn    condition     = ${if exists {/etc/exim4/domains/$domain/antispam}{yes}{no}}
          set acl_m1    = yes
.endif
  accept


acl_check_data:
.ifdef CLAMD
         deny message   = Message contains a virus ($malware_name) and has been rejected
         malware        = *
         condition      = ${if eq{$acl_m0}{yes}{yes}{no}}
.endif

.ifdef SPAMASSASSIN
  warn
         !authenticated = *
         hosts          = !+relay_from_hosts
         condition      = ${if < {$message_size}{100K}}
         condition      = ${if eq{$acl_m1}{yes}{yes}{no}}
         spam           = nobody:true/defer_ok
         add_header     = X-Spam-Score: $spam_score_int
         add_header     = X-Spam-Bar: $spam_bar
         add_header     = X-Spam-Report: $spam_report
         set acl_m2     = $spam_score_int
  warn
         condition      = ${if !eq{$acl_m2}{} {yes}{no}}
         condition      = ${if >{$acl_m2}{SPAM_SCORE} {yes}{no}}
         add_header     = X-Spam-Status: Yes
         message        = SpamAssassin detected spam (from $sender_address to $recipients).
.endif
  accept


acl_check_mime:
  deny   message       = Blacklisted file extension detected
         condition     = ${if match {${lc:$mime_filename}}{\N(\.ade|\.adp|\.bat|\.chm|\.cmd|\.com|\.cpl|\.exe|\.hta|\.ins|\.isp|\.jse|\.lib|\.lnk|\.mde|\.msc|\.msp|\.mst|\.pif|\.scr|\.sct|\.shb|\.sh|\.sys|\.vb|\.vbe|\.vbs|\.vxd|\.wsc|\.wsf|\.wsh)$\N}{1}{0}}
  accept

##########################################################################
begin authenticators
##########################################################################
dovecot_plain:
  driver = dovecot
  public_name = PLAIN
  server_socket = /var/run/dovecot/auth-client
  server_set_id = $auth1

dovecot_login:
  driver = dovecot
  public_name = LOGIN
  server_socket = /var/run/dovecot/auth-client
  server_set_id = $auth1

##########################################################################
begin routers
##########################################################################
#smarthost:
#  driver = manualroute
#  domains = ! +local_domains
#  transport = remote_smtp
#  route_list = * smartrelay.vestacp.com
#  no_more
#  no_verify

dnslookup:
  driver = dnslookup
  domains = !+local_domains
  transport = remote_smtp
  no_more

userforward:
  driver = redirect
  check_local_user
  file = $home/.forward
  allow_filter
  no_verify
  no_expn
  check_ancestor
  file_transport = address_file
  pipe_transport = address_pipe
  reply_transport = address_reply

procmail:
  driver = accept
  check_local_user
  require_files = ${local_part}:+${home}/.procmailrc:/usr/bin/procmail
  transport = procmail
  no_verify

autoreplay:
  driver = accept
  require_files = /etc/exim4/domains/$domain/autoreply.${local_part}.msg
  condition = ${if exists{/etc/exim4/domains/$domain/autoreply.${local_part}.msg}}{yes}{no}}
  retry_use_local_part
  transport = userautoreply
  unseen

aliases:
  driver = redirect
  headers_add = X-redirected: yes
  data = ${extract{1}{:}{${lookup{$local_part@$domain}lsearch{/etc/exim4/domains/$domain/aliases}}}}
  require_files = /etc/exim4/domains/$domain/aliases
  redirect_router = dnslookup
  pipe_transport = address_pipe
  unseen

localuser_fwd_only:
  driver = accept
  transport = devnull
  condition = ${if exists{/etc/exim4/domains/$domain/fwd_only}{${lookup{$local_part}lsearch{/etc/exim4/domains/$domain/fwd_only}{true}{false}}}}

localuser:
  driver = accept
#  transport = dovecot_lmtp
  transport = dovecot_lda
  condition = ${lookup{$local_part}lsearch{/etc/exim4/domains/$domain/passwd}{true}{false}}

catchall:
  driver = redirect
  headers_add = X-redirected: yes
  require_files = /etc/exim4/domains/$domain/aliases
  data = ${extract{1}{:}{${lookup{*@$domain}lsearch{/etc/exim4/domains/$domain/aliases}}}}
  file_transport = local_delivery
  redirect_router = dnslookup

terminate_alias:
  driver = accept
  transport = devnull
  condition = ${lookup{$local_part@$domain}lsearch{/etc/exim4/domains/$domain/aliases}{true}{false}}


##########################################################################
begin transports
##########################################################################
remote_smtp:
  driver = smtp
  #helo_data = $sender_address_domain
  #openssl_options = +no_sslv2 +no_sslv3
  tls_require_ciphers = NORMAL:!VERS-SSL3.0
  dkim_domain = DKIM_DOMAIN
  dkim_selector = mail
  dkim_private_key = DKIM_PRIVATE_KEY
  dkim_canon = relaxed
  dkim_strict = 0

procmail:
  driver = pipe
  command = "/usr/bin/procmail -d $local_part"
  return_path_add
  delivery_date_add
  envelope_to_add
  user = $local_part
  initgroups
  return_output

dovecot_lmtp:
  driver = lmtp
  socket = /var/run/dovecot/lmtp
  # Max no. of deliveries per request, default is 1
  batch_max = 200
  # Is default already false, but still handy to have here.
  ignore_quota = false

dovecot_lda:
  driver = pipe
  # Add -d $local_part@$domain to do userdb lookup (and add quota, home dir and such).
  command = /usr/bin/sudo /usr/lib/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain
  message_prefix =
  message_suffix =
  delivery_date_add
  envelope_to_add
  return_path_add
  log_output
  user = dovelda
  temp_errors = 64 : 69 : 70: 71 : 72 : 73 : 74 : 75 : 78

address_pipe:
  driver = pipe
  return_output

address_file:
  driver = appendfile
  delivery_date_add
  envelope_to_add
  return_path_add

address_reply:
  driver = autoreply

userautoreply:
  driver = autoreply
  file = /etc/exim4/domains/$domain/autoreply.${local_part}.msg
  from = "${local_part}@${domain}"
  subject = "${if def:h_Subject: {Autoreply: ${quote:${escape:$h_Subject:}}} {Autoreply Message}}"
  to = "${sender_address}"

devnull:
  driver = appendfile
  file = /dev/null

##########################################################################
begin retry
*                      *           F,2h,15m; G,16h,1h,1.5; F,4d,6h
##########################################################################
begin rewrite

Vesta Control Panel - Forum