DosGate changelog
Версия 3.8.0
Дата релиза: 10.03.2025
3.8.0 DosGate Core:
* Отслеживание адреса правила во время обработки пакета.
* Расширение заголовка UH для включения адреса правила при отбрасывании сетевого пакета.
* Добавлена дополнительная информация BPF для возможного аудита через API.
* Все новые функции из версии 3.7.4.
* Исправлены ошибки.
1.5.0 DosGate-UH:
* Добавлена поддержка JA4-TLS везде, где использовался JA3-TLS для
более точной классификации зашифрованного трафика с учетом новых особенностей
протокола TLS. Это улучшает идентификацию приложений и детектирование аномалий
по сравнению с JA3-TLS.
* Добавлен формат захвата pcap-ng.
* В дампах для каждого сброшенного пакета добавляется комментарий с
порядковым номером правила в профиле, что позволяет отслеживать, какие контрмеры отклоняют трафик.
* Добавлена возможность отключить управление прерываниями на сложных платформах через конфигурацию.
* Улучшены эвристики управления прерываниями в режиме non-MSI и при совместном
использовании векторов прерываний.
* Удалены статистики pbuf/seg. Теперь они управляются библиотекой.
* Исправлены ошибки.
Версия 3.7.4
3.7.4:
* Redesign rules from scratch for less complexity and stack usage
* Implement "strong" rule negation model
* Improve protocol lists processing from/to strings
* Add ability to negate protocol match
* Implement match merging for selected matches
* Split matches into protocol and specific parts and combine protocol
parts into one comparison code block. This allows to skip rule without
the need to process it in many cases
* Reorder matches within frame by their complexity. This allows
negative result to trigger earlier on low-complexity matches,
potentialy avoiding execution of high-complexity matches
* Implement parsing and caching of tcp options:
Move options processing to prolog bpf section to
reduce complexity of the main section
Fail parsing TCP on invalid options
Use cached tcp options where possible to reduce verifier
complexity
* Add "strict-check" daemon-level config element, which is a
comma-separated list of flags, each of which controls which additional
checks to perform at protocol parsing stage
* Implement validating of IPv4 options. This is optional feature
controlled by adding "ipv4-options" flag to the "strict-check" daemon-level
config element. IPv4 option validation is disabled by default
* Remove flow record flushing from match code, move it into arena
prolog. This moves complexity from important profile part to less
important prolog part, where complexity budget is much more relaxed
* Fix improper use of negative caching of prefixset lookups
* priority match for IPv4 ToS or IPv6 TC
* PRIORITY action to set IPv4 ToS or IPv6 TC
* Allow filtering marks on geoip country both in cli and api
* Add audit commant to arena cli/api to view bpf-specific program tags
(hashes) and bpf ids
* Bug fixes
Версия 3.6.0
12.08.2024 dosgate-core
* Add support for exporting frames in UH (cybert compatibility for bot management module).
EXPORT target marks frame for export just like CAPTURE target marks frame for capturing
* Implement tracing for noizy bpf code
* Implement redirect on a cpu set. It will use NIC-provided frame hash
if possible and fall-back to direct hash calculation if hash is not
available. Frames are distributed over target cpu set in a RSS-like
manner, so that flow integrity is not broken
* Update PASS action to allow moving traffic to other cpus
* Dosgate will refuse to run on a system with kernel
older than OS ABI level. For Ubuntu 22.04 this is 5.15.0
* Bug fixes
12.08.2024 dosgate-UH
* Export via libxskexp. Allows exporting both frame and TLS hello
messages
* Add event hook of application getting fully populated. For TLS
it is the moment of valid hello handshake message
* Implement tracing, and move all debug/dump into it
* Rework connection confirmation. For TCP it requires valid ack with
appropriate sequence number
* Add protocol-level stats
* Implement tls-level defragmenting of handshake protocol
* Event subsystem
* Get rid of offenders, superseded by events
* Proper yes/no/auto logic of zc mode to mirror what kernel expects
* Ability to select frame buffer size for older cards (some ixgbe
macs)
* Prepare for real bidirectional mode: conntrack pinning, conntrack
backlog, worker backlog
* Change logic for fill/comp queues to prepare for external sources if
there are any
* Do not rely on configuration for zc and redirection, instead read
kernel report of a socket bind to detect zero-copy mode, and only set
redirection if needed. Required for auto logic of zc mode
* Rework conntrack reclaim for pinninig
* Allow for conntrack isolation selection: by dosgate metrics
(arena + profile, arena, none), by vlan
* Allow for instant time-wait conntrack reclaim in case of collision with
new connection
* Allow to start a tcp connection with a SYN+ACK frame in unidirectional
* Better handling of icmp unreachables for tcp/udp conntracks
mode
* More metrics for TLS, to be used as fields in events
* Bug fixes
Версия 3.4.2-1
28.02.2024
* Новый метод TCP-авторизации за счет синхронизации TCP ISN. Поддерживаются 2 модуля-агента:
Dosgate-Module1:
Поддержка IPv4 и IPv6
RFC 7323, 2018, 3168
Корректный подсчет MTU
Только SIP hash
Beget-Module2:
Поддержка IPv4
SIP и SHA hash
* Изменена работа меток: hmark, dhmark, sdhmark, connmark.
Теперь, администратор видит сколько времени назад запись появилась в метке,
а также может использовать новые операторы (равно, выше чем, ниже чем, ...) с помощью
которых можно проверять не только диапазон value, но и сам установленный таймер
в секундах для каждой из записей.
Например, теперь можно проверить в какой конкретно момент IP-адрес был
заблокирован тем или иным правилом. Данный таймер можно хранить
вплоть до 1 года и до 100 млн записей, таким образом администратор системы имеет
полную историю по всем IP-адресам помещенным в метки вплоть до 100 миллионной записи или 1 года
по умолчанию.
* Добавлена поддержка драйверов ixgbe и virtio для dosgate_uh,
добавлена частичная поддержка vmxnet3
* В tcpopts добавлена возможность выбрать null (опции отсутствуют).
* Оптимизация побайтового мэтчинга (-m seq)
* Поддержка веб-интерфейса версии 3.9.9
* Другие исправления и нововведения по фич реквестам, баг-фиксы
Версия 3.2.3-3
21.09.2023
* QoL-обновления для ускорения работы сокета и работающего с ним веб-интерфейса в обновлении 2.5.1
Версия 3.2.3-2
01.09.2023
* bug fixes
Версия 3.2.3-1
* Изменен побайтовый поиск ('-m seq')
* Реализована возможность модификации размера таблиц данных. Подробнее: https://docs.dosgate.com/map_tuning/
* Добавлена возможность обработки и экспорта FLOW. Поддерживается лог-файл, BSD syslog, stderr/stdout и IPFIX
* Оптимизация функций системы, минорное увеличение производительности (до 5%)
* Минорные баг-фиксы
Версия 3.2.2-5
* more bug fixes: prefixsets, geoip, marks
Версия 3.2.2-4
* bug fixes related to -m seq function
Версия 3.2.2
* dosgate_uh release (1.0): conntrack, TLS validation, defragmentation, checksum,
support for TLS cipher-suite, JA3-fingerprints
* Try, to load geoip database from /etc/dosgate/GeoLite2-Country.mmdb
first an fallback to usual /usr/share/dosgate/GeoLite2-Country.mmdb on
error, just like fixups do. First one is supplied by admin, second is
supplied by distribution
* New action RETURN. Returns from current chain to the rule next to
GOTO if used in chain, or equivalent to ACCEPT when used in profile
* New action CAPTURE. Marks frame for capture right before
accept/drop. Works only when upper-half is active.
* Update PASS action. Add ability to pass to upper-half with optional
application and its profile id. Does nothing if upper-half is not
active
* Improve TCPAUTH. Make timeouts per-direction, modify greylist auth type
* Speedup large prefixsets (>10K entries) at the expense of some more
RAM
* Implement map key caching. Must speed up every match/action which
uses maps, that is almost all of them
* Speedup hmark cache lookup by splitting out id into separate linear
index
* Add daemon level command to show some internal stats via CLI
* Use early mark cache flushing in reply path. Replied packets are
modified and thus produce invalid mark keys, so need to flush before
modification takes place
* Use indirect address matching via static value/mask arrays to avoid
LLVM bs with stack and 'all when there are many operations alongside
constant immediates. Set transition from direct to indirect match at
8, and keep eye on it
* Implement prefixset lookup caching. It does not look extremely
useful except for one case. Geoip and prefixsets are essentially the
same, and geoip can be used multiple times to apply different policies
to different countries. In this case, there will be only one map
lookup
* Attempt to use "global methods" to hide complexity from verifier.
According to commit notes, global methods are verified once as a
single entity, and thus they should not produce execution paths from
their inside code, and should be able to remove existing parallel
paths as "merge points". Given all the limitations, first candidates
for converion are map key generation and mark cache lookups. Using
global methods for specific actions/matches proved impractical due to
LLVM messing up with stack and generating loads of code paths quickly
eating up instruction budget
* Use global methods for rules, as they do not suffer from limitations
and we can limit most of the code paths in the controlled manner by
adjusting rule limit per method, which is already implemented
* Improve tcp flags match by moving from legacy BSD's header layout with
bitfield for flags to type casting and matching by value/mask on a
4-byte word, as suggested by linux kernel's tcp.h
* Implement daemon state save/load via API. Add a field to context
section in daemon config, which holds context's save file name. States
are saved for contexts with non-empty save names as /var/lib/dosgate/state/<name>.
State loading is performed in reverse, for contexts which have
non-empty save name and with existing file
/var/lib/dosgate/state/<name>. State load is destructive, i.e. current
state is replaced with saved one. States are versioned and
backward-compatible, so newer dosgate can load older states, but older
dosgate will refuse to load newer state. State save is (and never will
be) atomic. State load via API/CLI is not atomic and never will be.
Atomic state load on startup is planned for future versions. State can
be moved freely across hosts, and adapted in-place by properly naming
state files according to config. All times, dates and timeouts are
exchanged by converting to wallclock time and date, and thus proper
time synchronisation via ntp or sntp is required. Only non-volatile
data missing from configuration is included in state, that is marks
atm. See doc/state.txt for details
* Add arena "id" field to config, required for upper-half
* Implement full api for chain target
Версия 3.2.1-4
* Bug fixes (memleak, ...)
Версия 3.2.1
* Use percpu LRU for marks and ratelimit maps
* Improve CLI listing for object statuses
* Implement advisory CLI locking. Locking is performed on a console
session basis. Read dgctl --help
* Implement regex-based listing filtering for profiles and chains
* Preliminary work for upcoming userspace processing
* Add support for bpf loading method in config. New "daemon" object
with "xdp-mode" parameter. Possible values are "auto" (default) - try
drv mode, fallback to skb; "drv" or "hw" - force drv mode, "skb", "sw"
or "generic" - force skb mode. Different errors may arise when switching
modes without forcefuly unloading program from interfaces
* More workarounds for BPF ALU32 and verifier issues
* Fixed known bugs
Версия 3.2.0
* Introduce split maps. There is only one way to handle overflow -
LRU. In case of a steady input flow, LRU becomes heavily contended and
lowers overall map performance to 1/n (n - number of cpus). To
overcome this there are two approaches. First one is to make LRUs
percpu. This shows good results, but map tends to stabilize at 1/2 of
capacity, which calls for doubling it wasting memory and
increasing cache miss ratio. Split maps use global LRU, but they split
load over multiple objects by hashing keys and selecting object based
on hash value.
* Min RAM is 8G now due to increased map size
* Make TCPAUTH maps split-aware as heavy contended in case of flood.
For now, we make TCPAUTH maps percpu lru. But it is possible to switch
to split if needed.
* Introduce named counters. Old counters can be used alongside named
ones for backward compatibility. Adding unnamed counter in place of
named will move later into unused slot, thus zeroing it.
* Remove all stats commands. Stats now live in their own level
* Introduce "RATE" action. Macro-level rate estimator with
conform/exceed/cooldown states. Rate estimation is performed on
intervals of ~100ms. Using exponential filtring, it can detect traffic
"spikes". Can use both current and weighted rates, or remembered
values at exceed switchover moments, and use percentage or exact
thresholds with min/max estimation.
* Initial inline support. Switch transparenly ARP, IPv6 mc/ll.
Controlled transparent switch of l2 multicast and
unknown traffic. Controlled tx/pass of l2 multicast and unknown
in bounce mode. LACP switch in transparent mode
* Unhandled l2 link-local multicast is always passed to OS. This includes
stp, gvrp, different kinds of proprietary loop-detection. Better
force net eng to revisit the scheme than fight with miracuolous side effects
* Ubuntu 22.04.1 LTS is the new target OS
* Reliable system identification, based on hardware. Strace
obfuscation. Can be retrieved with command-line key or via API
* New licensing system. Package/Admin/Operator managed licenses.
Add/remove/list via API. Check for valid system id. Preliminary support
for license constrains, not used atm
* Build system support for licenses. If not certificates are supplied
at build time, no license performed at start and run time. Otherwise,
build system forms a list of valid certificate fingerprints and embeds
it into daemon, obfuscating from reverse engineering.
* Licenses are created with dgadm, valid certificate, private key and
contents in JSON format. Certificate, used to create a license, must
be trusted and embedded into daemon at build time
* Licenses can be examined with dgadm. Outputs internal JSON document
* dgadm support to include licenses in backup or restore them
* All "commit"-class API commands require valid license
* New target in dgadm and some batch updates
* Configure interfaces in interface_set. Set promisc if any
switchtables entries are inline, and unconditionaly disabel vlan rx
offload
* Rewrite mem functions to pass validation with LLVM-13 and newer
kernels. Need more "strict" approach with "ctx-offset" pairs instead of
pointers, and split memmem into a series of non-inline methods to pass
"complexity" checks
* Change match_sequence to use new API. Sequence is
still too heavy, and only ~3 rules per profile are accepted.
Considering sequence _searching_ is already brain-damaged in
anti-ddos, this is fine. Checking sequence on a fixed offset should be
fine
* Change match_tcpopt for new API. Add matching on a given
sequence of headers, which is probably of more use for anti-ddos than
checking for option presence. Still, both checks are supported in
tcpopt match and can be used simultaneously
* New "passthrough" context type. Much lighter than arena, only
manages some stat counters and passes traffic to tx without
modifocations. No parsing or map lookups occurs. Should be useful for
returning traffic in inline switch mode. Stats from passthrough
contexts are sent to collectd and are available via API/CLI calls
* Bug fixes
Версия 3.1.3
* Fixed batch map operations according to undocumented kernel
behavior. These where visible in case of nearly-full batch operations
failing and ex. marks not being listed correctly. Also this case
can lead to improper map synchronization
* Fixed profile destruction segfault due to router map sync and
out-of-bounds memset
* Fixed memory leak in mapdiff
* Fixed listen socket accept logic, which processed only
one socket per event
* Update socket handling for simplicity and cleanup. This
also fixes segfault on EOF
* Update libdt and libaevent APIs
* Create internal API for error handling and convert (mostly)
everything to use it. This allows passing every error to the
JSON API user from any point of call chain
* Rewrite lib, prog_lib, prog_set, interface_set
for simplicity and cleanup. This also fixes the issue
with program tag not being updated after rebuild in attach
path and overall improves program map synchronization process
* Fix an issue with RX prog is not being marked dirty
if some switchtable entries have been removed
* TCP Auth: narrow down authentication to single
TCP session per host. Other sessions will get
invalid state until selected one either times out or
succeeds
* Implement rule chains and API for them
* "replace" command for prefixset. Completely replaces
prefixset contents with new data
* Save BPF program source code in library. BPF code is GPL licenced,
and thus its source code must be available. This behavior can be
disabled if needed
* Bug fixes
Версия 3.1.2
* API: GeoIP in mark listing
* API: profile rename and description edit
* profile description
* API: preliminary support for backref
* Automatic stats export via collectd
* API: stats target
* API: URL
* TCP Reverse Auth
* GeoIP
* Prefixsets
* Bug fixes
Версия 3.1.1
* Add table-based TCP Authentication methods
* Bug fixes
Версия 3.1.0
* Improve Full API
* Improve help for RATELIMIT action
* Rework DNS: full header support
* Rework frag: multiple fragment states in one rule
* Improve IPv6 parsing to properly handle malformed packets and
fragments
* Improve host marks: protocol deps have gone into helpers,
rules/matches are protocol-agnostinc now
* Preliminary prefix sets - match + external management, no API for
now
* Implement map operations with batching
* Add backup/restore to dgadm
* Cosmetics
* Bug fixes