580fe192df Merge pull request #3247 from airween/v3/master
24dbcfe637 Change release version to v3.0.13
1a8c96a1cd Merge pull request #3206 from airween/v3/release2408
b489fd3562 Format fix
2ab970be2e Finalize CHANGES
865b75b8fa Merge branch 'owasp-modsecurity:v3/master' into v3/release2408
542a5ea35c Added PR #3243
f180e647a1 Merge pull request #3243 from eduar-hte/valid-hex-fix
a4604b66f7 Added new tests to op @pm
27cc8edbfe Adjust reference to modsecurity::utils::string::VALID_HEX - This function (previously a #define) was previously in the global namespace and was moved into modsecurity::utils::string in commit a6d64bf.
358618951a Added PR #3240
9403cf6f5d Merge pull request #3240 from frozenice/patch-1
07fb580415 Added PR's #3233 and #3231
9148668571 Merge pull request #3231 from eduar-hte/remove-copies-transformations
4951702d45 Merge pull request #3233 from eduar-hte/remove-copies-pm-operator
3e9d8107a8 Removed multiple heap-allocated copies in parse_pm_content - The previous version of this function was doing three strdup copies to parse the pm content. The updated version only copies the value once (in order not to modify the Operator's m_param member variable), and then performs the updates inline. - Binary parsing was broken because digits were not compared as characters. - Fail parsing when an invalid hex character is found. - Error message in parse_pm_content would reference freed memory if accessed by caller. Removed anyway because it was unused.
a6d64bf615 Replaced VALID_HEX, ISODIGIT & NBSP macros in string.h - Moved them into modsecurity::utils::string to avoid polluting the global namespace.
2f5dac5c4c Simplified initialization of Transformation's action_kind - Some of the Transformation classes would initialize their Action's action_kind using the default (using Transformation constructor without an action_kind parameter). - Others, however, would use that constructor and initialize action_kind manually in their constructor, but setting the default value (RunTimeBeforeMatchAttemptKind = 1), which was redundant. - Removed unused Transformation constructor to specify action_kind. - Converted Action::Kind into an 'enum class' to require using the enum constants (instead of integer values, which are difficult to track in the codebase and change)
7023c0a8b4 Refactored sha1 & md5 utils to share implementation and reduce code duplication.
fedec96a7e Refactored base64 utils to share implementation and reduce code duplication.
34da8eeeee Pass RuleWithActions::executeTransformation arguments by reference - This function already expects these arguments not to be null pointers, doesn't validate them and just dereference them. - In order to make this explicit and enforced by the compiler, they're now passed as references.
b647dbd905 Remove unnecessary heap-allocation & copy in Transaction::extractArguments - utils::urldecode_nonstrict_inplace decodes inplace so key & value, which are values returned by utils::string::ssplit_pair can be just be modified and do not need to be copied. - Updated signature of utils::urldecode_nonstrict_inplace, as its two callers already have std::string values.
021d0caa33 Perform NormalisePath & NormalisePathWin transformations in-place
2c3c228725 Perform Utf8ToUnicode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation.
17a2cbd164 Perform UrlDecodeUni & UrlDecode transformations in-place - Use std::string in UrlEncode transformation, instead of manually memory management. This avoids an additional copy after completing encoding by just swapping the encoded value and the input. - Removed inplace helper function from the class, as it's only referenced by the implementation.
8bf4d96e6b Perform HtmlEntityDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation.
7d5c9faa43 Perform JsDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation.
a520369da0 Perform EscapeSeqDecode transformation in-place - Removed ansi_c_sequences_decode_inplace helper function from the class, as it's only referenced by the implementation.
727f2bf840 Perform CssDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation.
e687140d05 Perform HexDecode transformation in-place - Removed inplace helper function from the class, as it's only referenced by the implementation.
4670710376 Perform LowerCase & UpperCase transformations in-place - Refactored to share implementation and reduce code duplication.
fd8a979463 Perform SqlHexDecode transformation in-place - Validate buffer size before accessing data. The previous implementation would only check that there was a character available in the buffer but could continue processing/reading characters from an hex representation without checking bounds. - Removed inplace & mytolower helper functions from the class, as they're only referenced by the implementation. - Removed duplicate VALID_HEX & ISODIGIT macros, already in src/utils/string.h.
2915ee60e2 Perform Trim, TrimLeft & TrimRight transformations in-place
74d150c068 Perform RemoveCommentsChar, RemoveComments & ReplaceComments transformations in-place
da775eca81 Perform ReplaceNulls transformation in-place
1505025990 Perform RemoveNulls & RemoveWhitespace transformations in-place - Refactored to share implementation.
1236d9a7cd Perform CompressWhitespace transformation in-place
13203ae5e7 Perform CmdLine transformation in-place
3ff72fbbc5 Perform ParityEven7bit, ParityOdd7bit & ParityZero7bit transformations in-place - Refactored to share implementations of ParityEven7bit & ParityOdd7bit.
5d39890783 Updated Transformation::evaluate signature to allow for in-place updates, removing unnecessary heap allocated copies. - Renamed Transformation::evaluate to Transformation::transform to avoid confusion with Action's overload methods. - Updated Transformation::transform signature to receive the value by reference and perform the transformation inline, if possible. - Some transformations still need to use a temporary std::string to perform their work, and then copy the result back. - Made Transformation::transform methods const and updated Transaction parameter to be const. - Transaction parameter could not be removed because it's used by just a single transformation, UrlDecodeUni. - Removed std::string Action::evaluate(const std::string &exp, Transaction *transaction); which was only implemented by Transformation but was not used from the base class, but only after downcasting to Transformation, so it can just be declared there (and not pollute other actions with a default member implementation -that does nothing- which is never called).
094143801a Added PR #3232
97c8766ef1 Merge pull request #3232 from eduar-hte/failed-unit-tests-automake-output
315b3d6e77 Lua::run: Move logging of `str` parameter to higher log level.
df081af870 Adjusted pthread LDFLAG in examples required for multithreading. - Some versions of gcc/libc require setting the pthread flag when using std::thread, which to implement it. - This was found compiling the library in a Debian (bullseye) container.
a5f223cb52 Individual test result should not be printed for automake output - Test results output escape characters to highlight whether the test passed or failed. Additionally, the input & output for each test can include non-ASCII characters. These characters break parsing of results (.log & .trs files) with grep, as the files are interpreted to be binary.
752ab76238 Merge pull request #3229 from eduar-hte/pthread-makefile
2cb1d032e3 Restore pthread LDFLAG.
71bea86e91 Added PR #3228
b4f52325bd Merge pull request #3228 from eduar-hte/asctime-multithread
746f4d7e80 Added PR #3227
554bd30e74 Merge pull request #3227 from eduar-hte/pm-operator-multithreading
c9af0c747e Merge pull request #3221 from eduar-hte/unittest-multithreaded
c4b2723a4f Added PR #3225
a6b287e120 Merge pull request #3225 from airween/v3/mpinvcharreqbody
ee5f95eb04 Added support to run unit tests in a multithreaded context - This is controlled by specifying the 'mtstress' argument when running `unit_test`. - The goal is to detect if the operator/transformation fails in this context. - In this mode, the test will be executed 5'000 times in 50 threads concurrently. - Allocation & initialization of the operator/transformation is performed once in the main thread, while the evaluation is executed in the threads. - This is consistent with the library's support for multithreading, where initialization and loading of rules is expected to run once. See issue #3215.
23a341eb6a Calculate sizes of strftime buffers based on format strings - Leverage std::size to determine buffer size at compile time. - Simplified 'TimeMon::evaluate' implementation as it was using strftime to get the month, convert the string to int, and then decrement it by one to make it zero based. This same value is already available in the 'struct tm' previously generated with the call to localtime_r (and where the month is already zero-based)
5e6fcbc60b Replace usage of std::ctime, which is not safe for use in multithreaded contexts - std::ctime returns a pointer to a string that "may be shared between std::asctime and std::ctime, and may be overwritten on each invocation of any of those functions.". - https://en.cppreference.com/w/cpp/chrono/c/ctime - Replaced with call to strftime to generate the same string representation (using the format string: %c) - Leveraged localtime_r (which is thread-safe) to convert time_t to struct tm, as required by strftime.
8d6b185856 Removed unnecessary lock to call acmp_process_quick in Pm::evaluate - This was introduced in commit 119a6fc & 7d786b3 because of a potential issue reported in #1573. - The ACMP tree structure is initialized when the operator is initialized. - During transaction execution the ACMP tree structure is only 'read' while traversing the tree (in acmp_process_quick) so this is safe for use in a multi-threaded environment.
f42bc38f4c Update CHANGES
718d121ee3 Merge pull request #3216 from eduar-hte/inmemory-collection-shared-mutex
32f6f78e78 Merge pull request #3222 from eduar-hte/remove-copies
6388d88f38 Check if the MP header contains invalid character
77adb57524 Avoid std::string copy in ssplit argument - Other minor changes reported by sonarcloud
305f33fea1 Merge pull request #3224 from airween/v3/sethostnametestfix
eb26b7960c Fix regression test result; Add test to main test-suite list
cc0f893854 Removed unused overload of dash_if_empty that sonarcloud flags as potential buffer overflow
8b17f3691f Inline string functions
1534ee2448 Removed unnecessary copies
f8dd09f7c9 Avoid creating a new std::string on the heap to create VariableValue - Introduced helper method addVariableOrigin to reduce code duplication.
bb07de9ad7 toupper/tolower is already receiving a copy, so it doesn't need to create a new one to transform it - Make functions inline to improve performance - Introduced helper method toCaseHelper to remove code duplication
4bf9616f9e Adding multithreaded example from issue #3054 (by airween) - Rewritten to use C++ libModSecurity API and std::thread (instead of pthreads)
293cd214c7 Removed usage of pthreads and replaced with std C++ features - Replaced pthread_mutex_t in modsecurity::operators::Pm with std::mutex - Replaced pthread's thread usage in reading_logs_via_rule_message example with std::thread. - Simplified and modernized C++ code. - Removed unnecessary includes of pthread.h
4e15f9ef71 Turn off LMDB by default in Windows build to align with defaults for other platforms - Replaced WITHOUT_XXX build options with WITH_XXX to make it easier to understand and configure. - Updated GitHub workflow to align with these changes and include a build 'with lmdb' (again, analogous to non-Windows configurations)
e2b3c9594f Prevent concurrent access to data structure in resolve methods - As reported in #3054, the resolve methods in InMemoryPerProcess are not acquiring a lock/mutex to prevent concurrent access to the data structures that may be modified at the same time from other threads, and thus triggering undefined behaviour. - Replace inheritance of std::unordered_multimap data structure with data member to prevent potential clients to use it without acquiring the mutex to protect concurrent access. - Replace pthreads lock with std C++11 std::shared_mutex - Provides exclusive/shared lock access so that multiple readers can access the data at the same time, but only one writer. this is used to favor query performance by allowing more concurrent access to the data until an update needs to be performed. - Simplifies acquisition and disposal of lock/mutex with std::lock_guard, which has RAII semantics. - NOTE: Because std::shared_mutex is not recursive, calls to another function that tries to acquire the lock will fail. Introduced __store & __updateFirst helper methods to workaround this. - Updates to InMemoryPerProcess::resolveFirst - Updated the code to store the expired var in 'expiredVars' to delete them after iterating over the range (and releasing the read lock, as 'delIfExpired' needs to acquire it for exclusive access), as the current call to 'delIfExpired' would invalidate the range triggering undefined behaviour on the following iteration. - Noticed that in commit 118e1b3 the call to 'delIfExpired' in this function is done using 'it->second.getValue()'' instead of 'it->first', which seems incorrect (based on similar code in other resolveXXX functions). - Updated InMemoryPerProcess::delIfExpired to use 'std::find_if' (with a lambda that matches both the key and the 'isExpired' condition) because the data structure is a multimap. The version introduced in commit 118e1b3 could find an entry (not necessarily the first, because the map is unordered) where 'isExpired' is 'false' and exit, while another entry could be expired.
c575dce3d3 Added PR 3218, 3219, 3220
7bdc3c825c Merge pull request #3220 from eduar-hte/string-null
3a83196a71 Merge pull request #3219 from eduar-hte/cpp17
6f0e566f98 Merge pull request #3218 from eduar-hte/remove-dynamic-casts
c917d6a2dc Initialize variable in if statement to avoid doing dynamic_cast twice - Refactored duplicate code in RuleWithOperator::getVariablesExceptions - Leveraged auto to simplify declaration of dynamic_cast pointers.
18378c10f8 Removed unnecessary dynamic_casts
09980324a7 Added PR #3114
a23e88f79f Merge pull request #3114 from airween/v3/sonarmemleakfix
30a68de92d Creating a std::string with a null pointer is undefined behaviour. - cppreference mentions this about the constructor that receives a const char *: - Constructs the string with the contents initialized with a copy of the null-terminated character string pointed to by s. The length of the string is determined by the first null character. The behavior is undefined if [s, s + Traits::length(s)) is not a valid range (for example, if s is a null pointer). - C++23 introduces a deleted constructor to prevent this in static scenarios, which is how this issue was detected.
59254fe3bd Simplifiy configuration to build libModSecurity with std C++17 - Leveraged autoconf again to check whether the C++ compiler supports the required standard version and build using it. - Replaced the outdaded `ax_cxx_compile_stdcxx_11.m4` macro with the latest version of `ax_cxx_compile_stdcxx` which supports C++17. - https://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx.html - https://raw.githubusercontent.com/autoconf-archive/autoconf-archive/e4e5269db2764b9f53d759c24750ac6ca38e02ea/m4/ax_cxx_compile_stdcxx.m4 - This should also streamline updating to C++20 in the future.
546ec8fe9a Added PR #3217
a519c65902 Merge pull request #3217 from gberkes/v3/sonarcloud_Replace_this_declaration_by_a_structured_binding_declaration
13cce62b0b Added PR #3212
1d6e72e8e2 Merge pull request #3212 from eduar-hte/defensive-intervention
cf643d6072 Avoid duplicate definition of --enable-assertions=yes configure flag on Unix builds - This configuration flag was introduced in commit d47185d in the context of PR #3207. - Moved to the configure step's 'run' command in order to be shared across configurations. - For the sake of reference, matrix.platform.configure should be used for configuration flags that are needed for a specific platform/architecture (which was the reason it was introduced in commit d9255d8, PR #3144).
c50a397a87 Suppress cppcheck false positive unassignedVariable warning.
35e825d643 Refactor: replaced 3 declarations with 3 structured binding declarations.
5403b3d01c Update CHANGES; added newest PR's
e8db92ebb0 Merge pull request #3214 from gberkes/v3/Use_the_init-statement_to_declare_pos_inside_the_if_statement
a3ffc5a0d2 Merge pull request #3213 from gberkes/v3/sonar_move_these_3_includes_to_the_top_of_the_file
ab78d4af79 Refactor: used the init-statement to declare "pos" inside the if statement.
c46f470d6b Refactor: moved 3 #include directives to the top of the file.
0feaeacce5 Merge pull request #3211 from eduar-hte/secremoterules-regression
c802b46b7e Simplify parser error detection in testcase - After the GitHub macOS runner images were upgraded to macOS 14.6 (Sonoma), the test 'Include remote rules - failed download (Abort)' started failing because the error message reported by curl/OS is no longer 'HTTP response code said error'.
0b5493d4e7 Minor performance improvements setting up intervention's log - Initialize `log` temporary value on construction instead of doing default initialization and then calling `append`. - Leverage `std::string_view` to replace `const std::string&` parameters in `utils::string::replaceAll` to avoid creating a `std::string` object (and associated allocation and copy) for the string literal`%d`
c947f5e40d Do not assume ModSecurityIntervention argument to transaction::intervention has been initialized/cleaned - Keep m_it->disruptive value and use it as return value to guarantee that the value is correct. - If m_it->disruptive is false and the 'it' argument has not been initialized/cleaned, the function may incorrectly return a non-zero value. - When a disruptive intervention is being reported by the function, defensively initialize log & url to NULL if there's no such data to provide to the caller. - If the caller has not initialized/cleaned those fields in the 'it' argument, after returning from transaction::intervention, the user can safely read the log & url fields and in all scenarios they'll have valid values.
80dd45703b Update CHANGES - added PR 3210
68d551c5f9 Merge pull request #3210 from eduar-hte/shared-files-deadlock
ff303c761f Add newest changes
630751eee6 Merge pull request #3209 from eduar-hte/cleanup_api
8ec69bedd0 Merge pull request #3208 from eduar-hte/macos-apple-silicon
4b5f719906 Fixed shared files deadlock in a multi-threaded Windows application - The shared files Windows implementation introduced in PR #3132 works in multi-process single-threaded contexts but it doesn't work correctly in single-process multi-threaded contexts. - The issue is that the LockFileEx Win32 function works on a per-handle basis. - In a multi-process context, each process will have called SharedFiles::add_new_handler when initializing the SharedFile and obtained a handle, and thus locking will work. - When running ModSecurity in a single process using multiple threads, the initialization of the SharedFile will happen once and the handle will be shared by all threads. Then, if two threads try to write to the same shared file concurrently, they may deadlock as one of them will lock the file (by calling LockFileEx) and then proceed to write to the file. If before writing to the file and unlocking it, another thread calls LockFileEx on the same handle, the attempt to write to the file will lock generating a deadlock. - The new implementation replaces usage of LockFileEx/UnlockFileEx with a named mutex to lock access to the shared file. - A named mutex is used to support multi-process scenarios. - The mutex name is generated using the filename to support multiple shared files (such as that for the debug and audit logs). - This assumes that both process will initialize the SharedFile instance using the same filename (which is expected as they'd be using the same configuration file)
0dce46062b Fixed potential memory leak when there is an intervention and log or url is set.
dab9bb6a11 Added methods to free buffers allocated by ModSecurity APIs - The following methods are introduced to allow clients of libModSecurity that are not able to link and call the C/C++ standard library to be able to free the buffers allocated by libModSecurity. - msc_intervention_cleanup: Frees the buffers in a ModSecurityIntervention structure that have been allocated by calls to msc_intervention. - msc_rules_error_cleanup: Frees an error message buffer allocated by the msc_rules_xxx functions to detail the condition that triggered the error.
e31ff7e60b Build on macOS 14 arm64
6cffa8f904 Add _putenv() in case of WIN32 port instead of setenv()
82801752d4 Merge branch 'v3/master' into v3/sonarmemleakfix
2048730012 Update CHANGES
f04dcc0262 Merge pull request #3207 from gberkes/v3/remove_this_throw_call_transaction_h_mk2
b4cb24327c Fixed extra whitespace.
dc3f80a155 Fixed missing whitespace.
d47185d771 Build System: Introduce Configurable Assertion Handling
053e3b5266 Document the usage and the importance of assertions.
b4659959cd Refactor: Ensure safe error handling by removing isolated throw; statements.
5fa470189b Add PR's to CHANGES
b6d218f72d Merge pull request #3116 from gberkes/v3/remove_this_conditional_structure
97c3d15f31 Merge pull request #3203 from airween/v3/sethostname
c7efeb6d06 Merge branch 'owasp-modsecurity:v3/master' into v3/sethostname
a14cdc4ff5 Merge pull request #3182 from airween/v3/readmeupdate
4b38435a6e Merge pull request #3117 from airween/v3/eualrangebyfind
6449310831 Fix typos
937fc5ae59 Provide a function to set 'hostname' field in log
adba86e2bd Merge pull request #3185 from eduar-hte/git-describe
de8646e383 Merge pull request #3189 from bitbehz/fix/typo-build-win32-dockerfile
4fce2e3c1d Fixing typo in Dockerfile
f4d35383e9 fixed typo
5fe777aeb8 simplify submodules checkout (but fetch tags for git describe to work)
3dda900ee9 Merge pull request #3164 from eduar-hte/variable-origin
eb62cac7fa Add script to download OWASP CRS v4 to run benchmark - Simplified clone & checkout of CRS repository - Removed no longer maintained OWASP Core Ruleset v2
6faf6d7ec0 Removed unnecessary usage of heap-allocated VariableValue (m_var) - Removed unused methods
dc0a06fc70 Improve performance of VariableOrigin instances - The previous approach would create a std::unique_ptr and store it in a std::list in VariableValue (Origins) - The new approach now stores Origins in a std::vector and constructs VariableOrigin elements in-place on insertion. - Instead of having two heap-allocations for every added VariableOrigin instance, this performs only one. - If multiple origins are added, std::vector's growth strategy may even prevent a heap-allocation. There's a cost on growing the size of the vector, because a copy of current elements will be necessary. - Introduced reserveOrigin method to notify that multiple insertions will be made, so that we can use std::vector's reserve and do a single allocation (and copy of previous elements), and then just initialize the new elements in-place.
8d06e4c47b Typo fixes
baa7e694f9 Typo fixes
714e531134 Content improve
30ddc32c6f Update README.md: use submodule and use benchmark tool
7c174e95fa Merge pull request #3161 from eduar-hte/others-update
a3f40ef03c Replace Mbed TLS source code in repository with a submodule - Updated to latest Mbed TLS version (v3.6.0)
7732b5e8f3 Update libinjection to version v3.9.2-92-gb9fcaaf
dc8af8f23d Merge pull request #3160 from fzipi/v3/add-pull-request-template
98c672ddb4 chore: add PR template
2fd45f870b Merge pull request #3146 from eduar-hte/seclang-scanner-nounistd
37776fd262 Merge pull request #3144 from eduar-hte/gh-workflow-updates
d9255d85ca Updated GH Unix build configurations - Added support to build 32-bit versions of libModSecurity on Linux - Added support to build libModSecurity using clang on Linux (both 64-bit and 32-bit versions) - Fixed macOS dependencies to include yajl, not only because it is a required dependency, but because tests were not being run on macOS builds without it. - Added build 'without libxml' to Linux & macOS configurations. - Added build 'without ssdeep' to Linux configurations (already in macOS configuration) - Added build 'with lmdb' to Linux & macOS configurations, replacing the existing one 'without lmdb' because by default LMDB is disabled if not explicitly turn on in configure. - Removed 'without yajl' build because it's a required 3rd party dependency. - Added bison & flex dependencies to enable parser generation.
5a543d9c0b Updated .gitignore to ignore files generated in builds - build/win32/* files from Windows builds, other files from Unix builds
9e44964dc7 Use SRC_DIR argument
fbaf052a0a Update Windows build information after PR #3132
d0108efbc3 Update actions/checkout version to avoid deprecation warnings on GH workflow
636cf43d5e Separate workflow to run check-static (cppcheck) build step
7267c1dc21 Added support to run regression tests without libxml2 - Annotated regression tests that depend on libxml2 support - Added Windows build without libxml2
2c488386c4 Add options nounistd & never-interactive to seclang-scanner.ll - The parser is not used interactively so we can avoid including unistd.h, which is not available on Windows MSVC C++ compiler. - The #ifdef WIN32 introduced in PR #3132 would probably be overwritten when the parser is updated.
124a434439 Merge pull request #3141 from rkrishn7/v3/master
71a786b1e5 Merge pull request #3132 from eduar-hte/windows-port
1b2de5a5d3 Add support to turn 3rd party dependencies off - By default, all the 3rd party dependencies are enabled. - A dependency can be turned off by adding the "-DWITHOUT_xxx=ON" to the call of vcbuild.bat - List of 3rd party dependencies and associated option to turn them off: - LMDB: WITHOUT_LMDB - LUA: WITHOUT_LUA - LibXML2: WITHOUT_LIBXML2 - MaxMind: WITHOUT_MAXMIND - cURL: WITHOUT_CURL
6bf78f2560 Added GitHub workflow to build libModSecurity on Windows.
e6e2989bd5 Configure test fixture using CTest for Windows build - Added new test/test_suite.in with list of regression and unit tests previously in Makefile.am, to be shared between Unix and Windows builds. - Updated regression.cc & unit.cc to return the number of failed tests to indicate to CTest that the test failed. Similarly, a crash or unhandled exception terminates the process with a non-zero exit code. - This change doesn't affect running the tests with autotest in Unix builds because this processes test output from custom-test-driver & test-suite.sh, and ignores the exit code of the test runner. - Removed comment in test/test-cases/regression-offset-variable.json as this is not supported by JSON and prevents strict parsers to read and process the file. - Minor change in regression.cc's clearAuditLog to replace std::ifstream with std::ofstream as the mode to open the flag applies to an output stream. - Minor change in unit.cc to simplify code that deletes tests. - Minor changes to test/custom-test-driver to correct usage information.
4b8c3679b9 Add link to Rust bindings in README (#1)
a8e132f3a1 Replaced the use of "new" in find_resource - Addresses SonarCloud issue cpp:S5025 (Memory should not be managed manually) - This function was not changed for the Windows port, but a similar change to the one suggested was done in expandEnv in the same file. - The first stream is not destructed at the exact same point it was in the previous code (but rather when the second stream replaces it on assignment to the same variable). An arbitrary scope could have been introduced to destruct the object at the same place, but it doesn't seem to be necessary and would make the code a bit strange.
b69405a372 Use default keyword to implement constructor/destructor - Addresses SonarCloud cpp:S3490 issue (Special member function should not be defined unless a non standard behavior is required)
411bbb2d36 Updated case of winsock header files - Address SonarCloud cpp:S3806 issues ("#include" paths should be portable) - This is not an actual issue in this case, because WinSock2.h and WS2tcpip.h are Windows only.
faae58eed7 Added Windows build scripts using Build Tools for Visual Studio 2022 (MSVC compiler & CMake) and Conan package manager - Included Dockerfile to automate the setup process of prerequisites and build of libModSecurity binaries.
d7c49ed590 Added support to lock files on Windows and major rewrite to reintroduce reference counting and remove unused code. - In Windows build, replaced usage of fcntl with cmd F_SETLKW with Win32 APIs to do file locking (LockFileEx & UnlockFileEx). - Reintroduced the reference counting initially present in the class which is necessary to correctly handle merging of rules. This allows for correctly closing the file and removing the associated entry from m_handlers when the file is no longer used. - The need for reference counting can be seen in the example simple_example_using_c, where rules are initially loaded locally and then further rules are loaded remotely. This will initially open a shared file for a log, then in order to merge rules, the shared file is opened again for the new configuration. Then, the previous configuration closes the shared file on destruction. That is, two consecutive opens are done on a shared file, which is followed by a close. If the shared file is not reference counted, the shared file will be closed while there is still a reference active. The current version works because closing of the file has been disabled after reference counting was removed. - Replaced `std::vector` data structure with `std::unordered_map` to improve lookup/update times, and simplify code. - Removed unused code - Shared memory to store msc_file_handler structure - Initially SharedFiles used shared memory to store information about each shared file, including its file pointer and a mutex to synchronize access to the file on write. See code at commit 01c13da, in particular, usage of lock & fp fields in the msc_file_handler_t structure. - At that time, msc_file_handler_t included reference counting too with the using_it field, which was incremented when a file was opened and decremented on close. If the reference count reached zero, the shared file would be closed, the lock destroyed and the file handler entry removed from m_handlers. - Reference counting was removed in commit 7f9cd76, which introduced the following issues in SharedFiles::close: - No longer closes the file pointer. - The file pointer appears to be reset when a.second = 0, but this is a local copy of the data pair obtained from m_handlers, so this is essentially a nop (updating a local variable that is not referenced later in the function). - NOTE: The file pointer was moved out of the shared memory in this commit too, and stored alongside the msc_file_handler_t instance in the m_handlers entry associated to the shared file. - The lock is no longer destroyed. - The shared memory is marked to be destroyed in the call to: shmctl(a.first->shm_id_structure, IPC_RMID, NULL); - The shared file entry is not removed from m_handlers, so: - the file pointer is still valid, which is how writing to the file continues to work, - the reference to the shared memory is also present and will be marked to be destroyed whenever close is called again on the shared file. - File locking using the mutex in msc_file_handler_t was replaced in commit 3d20304 with usage of fcntl with cmd F_SETLKW. - At this time, it appears that the shared memory is no longer used, as the file pointer and locking no longer depend on it. - MODSEC_USE_GENERAL_LOCK - This code is introduced commit 7f9cd76 and is enabled if MODSEC_USE_GENERAL_LOCK` is defined. - The define is commented out in the source code since the original commit and is not present in the build configuration either. - In commit ff9152e, in the SharedFiles constructor, the initialization of the local variable toBeCreated is removed. This means that in this version, if MODSEC_USE_GENERAL_LOCK is enabled, execution of the code that checks on toBeCreated is undefined. - Then, in commit 9b40a04, the variable toBeCreated is initialized again, but is now set to false, which means that if MODSEC_USE_GENERAL_LOCK is enabled, the shared memory and lock it uses will *not* be initialized and thus doesn't work (execution of the current version will result in trying to acquire a lock that will be null). - I conclude that the feature is not used and can be removed. - Additionally, if it were working, I think the lock should be used in SharedFiles::write as well, which is a reader of the underlying data structures protected by this lock when they're modified in SharedFiles::open & SharedFiles::close.
50e78331b1 Updated Env::evaluate to support case-insensitive environment variable names in Windows - Env::evaluate - Environment variable names in Windows are case-insensitive, so in the Windows build we use strcasecmp to ignore case when matching variables in transaction->m_variableEnvs. - If the variable is found, we use the expected variable name to create the VariableValue instance, as further rule processing will look for the variable using case-sensitive comparisons. - This code is not limited to Windows to avoid another #ifdef block because for other platforms, because the env variable names are case-sensitive the value from either x.first and m_name will be the same. - In Windows build, avoid redefining environ, already defined by including stdlib.h.
50c35345ed Fixed use after free in ModSecurity::processContentOffset - Use after free issue detected with Address Sanitizer while running the reading_logs_with_offset example. - Keeps reference to last element in vars vector with vars.back(). Then it removes the element from vars calling vars.pop_back() which invalidates the reference, but it's accessed later in the function.
7bff2f77aa Updated references to coreruleset repository - For OWASP v2 rules, switch to a v2 tag for the paths referenced in the rest of the script to apply.
fef419f986 Minor changes related to std::shared_ptr usage in RuleWithActions - RuleWithActions::evaluate(Transaction *transaction) - Removed temporary rm local variable used to immediately create std::shared_ptr<RuleMessage>. - Leverage std::make_shared & auto to simplify code.
10c6ee726f Added support for expandEnv, createDir & cpu_seconds on Windows - expandEnv on Windows uses POCO C++ Libraries implementation of Glob - Paths of matched files are adjusted to preserve UNIX path separators for consistency with the rest of the code. - Minor change to code shared with other platforms that removes allocation of std::ifstream on the heap to check whether the file can be opened, which can be done with local stack variable that closes the file when out of scope. - createDir uses _mkdir on Windows, which doesn't support configuring the new directory's mode. - added public domain implementation of clock_gettime for clock_id CLOCK_PROCESS_CPUTIME_ID from mingw-w64's winpthreads to support cpu_seconds on Windows. - Updated included headers to support compilation on Windows (using Visual C++)
ebf1f8fd28 On Windows use the operating system's native CA store for certificate verification of https requests. - Updated included headers to support compilation on Windows (using Visual C++)
91a736692a Minor changes to debug_log_writer - Removed unused m_first data member. - Explicitly delete copy constructor and assignment operator. - Removed unused included headers.
373633ffe2 mkstemp is not available in Windows build, replaced with _mktemp_s plus _open. - Updated included headers to support compilation on Windows (using Visual C++) - Minor change to use C++ default (zero) initialization instead of calling memset.
35949179a4 setenv is not available in Windows build, replaced with _putenv_s
abbd7b2f42 Replaced usage of apr_snprintf with snprintf (already in Windows exclusive code block) - updated included headers to support compilation on Windows (using Visual C++)
942c8ba606 Replaced usage of usleep (not available in Visual C++) with C++11's std::this_thread::sleep_for & std::chrono::microseconds. - disabled build error from warning C4716 because process_request does not return a value and Visual C++ doesn't support [[noreturn]]
a48856822c Updated included headers to support compilation on Windows (using Visual C++) - most of posix related functions and constants in unistd.h can be found in io.h in Visual C++ - introduced src/compat/msvc.h to adjust for compiler differences (and avoid updating code with #ifdef blocks for Windows support) - removed some included headers that are not needed (both on Unix and Windows builds)
c8056483f7 Merge pull request #3134 from eduar-hte/inline-cppcheck-suppressions
1f419bba8f Implement sonarcloud suggestions
9f5dc200ba Replace final three suppressions entries with line numbers - These were initially not included in these changes, as they were other PRs (#3104 & #3132) that address them.
95ce3a7db4 Removed unused suppressions
7a9c0ab15f Removed unused suppresion and avoid copy of logPath
4aad8e0d06 Inline cppcheck suppressions
0c38023b21 Removed unmatchedSuppression entries
cd2dded659 Removed unnecessary break after return
0cd2f459f3 Address cppcheck suppressions in lmdb
94b68b2514 Minor updates to simplify code and remove cppcheck suppressions
fde9d279b0 Removed unnecessary cppcheck suppression and r-value reference as copy should be avoidded by RVO
b872f11f68 Fixed memory leak in examples/reading_logs_via_rule_message
4288f5a009 Enable inline suppressions in cppcheck
07e5a7058b Merge pull request #3128 from fzipi/update-submodules
7e085ffb6e fix: update submodule url
6217b4ec1e Merge pull request #3127 from fzipi/fix-rbl-check
30fe6f935b fix(rbl): typo in rbl check selector
16d0df0ff9 Optimized variable handling
7c4dcdfa4b Changed 'euqal_range()' + loop by 'find()' in resolveFirst() methods
5802626437 Deleted redundant code in 'ModSecurity::serverLog(...)'.
cdb6b839e1 Replaced variable type to 'auto'
1b6b631617 Use make_unique to create unique ptr
ca7c0ae0b9 Code refactorization
4085ff5536 Replace putenv by setenv
5b2404e44d Fix memleak in rules-check.cc
39848e5564 Fix memleak in regression.cc
625f9a5300 Merge pull request #2898 from brandonpayton/add-editorconfig
6d719bee5b Merge pull request #3016 from M4tteoP/uri_decode_invalid
5a6a53859a Merge pull request #3101 from airween/v3/updatechanges
387c4727f5 Update CHANGES
3fe51179ee Update CHANGES
97687496e9 Merge pull request #3098 from devzero2000/ep/scoped-for
2daebc090f src/utils/acmp.cc: reduce the scope of variable in a for () loop
7fed599fdb src/request_body_processor/multipart.cc: reduce the scope of variable in a for () loop
b23abf440a src/operators/verify_cc.cc: reduce the scope of variable in a for () loop
9842b92bd1 src/actions/transformations/hex_decode.cc: reduce the scope of variable in a for () loop
734646dbf1 Merge pull request #3096 from gberkes/v3/sonar_return_never_will_be_executed
64dfe41cec Refactor: Use pthread_exit(nullptr) instead of pthread_exit(NULL) for type safety.
f474cc59b8 Merge pull request #3079 from MirkoDziadzka/mirko-bump-c++-version
b3a267d506 Merge pull request #3095 from airween/v3/buildmacros
bf6042dd16 Marked the process_request() function with the [[noreturn]] attribute, as pthread_exit() does not return.
7b56db1811 Clean up 'return' never will be executed.
d0f3cf1389 Replace obsolete macros
985cb946cc Merge pull request #3092 from airween/v3/secstatusengoff
d7ef620e8b docs: Add info about modification
f850932f83 fix: Change 'SecEngineStatus' to Off by default
d0e6c163fc Merge pull request #3087 from fzipi/patch-2
626c12f1d4 Merge pull request #3086 from fzipi/patch-1
c63a9d6341 chore: update bug-report-for-version-2-x.md
9502b4980a chore: update bug-report-for-version-3-x.md
4e4f3291ad Merge pull request #3080 from airween/v3/testfixes
16c899fb3b Revert "Add new condition to test case"
80e2443884 Add new condition to test case
ed811f1062 Logical, syntax and cosmetic fixes on test cases
367a871f30 Bump the C++ version from C++11 to C++17
ca5f5163b4 Merge pull request #3027 from StarryVae/comment-spell-fix
3f8de775f9 Fix small comment spell
fcf205d599 fix: makes uri decode platform independent
6ad665f6bb Add editorconfig to help OSS contributors
git-subtree-dir: src/deps/src/modsecurity
git-subtree-split: 580fe192df1694934f93e3e008058996ea479e35
ea09d92adf bugfix: failed to build on windows.
aad278bc8c tests: bumped the NGINX core to 1.27.1.
d6cf113d6b feature: update nginx to v1.27.1.
f44c188590 bumped version of lua-nginx-module to 10027.
0cc05a6090 tests: t/188-*.t: use random port instead of 8090 port.
2d8fcbe643 tests: skip t/163-signal.t in check leak mode.
f1cbe4ae94 tests: t/020-subrequest.t: replace the random port with an unused five-digit port.
0ea5e94a80 tests: added curl_error for http/3.
b3c6aebcc7 travis: bumped the NGINX core to 1.27.0.
bf4bdcd5b2 bugfix: fixed keepalive error in cosocket.
5777a36a93 bugfix: failed to build on the old nginx version.
0b5507a255 bugfix: added initialization.
d535753f54 bugfix: nginx crashed when binding local address failed from lua.
29fe7a504f dev: util/build.sh: fixed command line argument validation and environment variable usage.
6768721255 doc: update lua-cjson ref link.
ed8cb8fe30 bugfix: treat shdict entries with ttl equal to 0 as expired.
6f311f82c3 tests: update ngixn to 1.25.3.
8670e53ea7 bugfix: let `balancer.recreate_request` API work for body data changed case.
45c63cda41 feature: add ssl trusted certificate.
94f55f7a4d bugfix: respect max retry after using balancer pool.
0a1c704c2e feature: support ngx.location.capture and ngx.location.capture_multi with `headers` option.
39d165ca41 bugfix: undefined symbol SSL_client_hello_get0_ext when linking against libressl.
6477a7b46d bugfix: fixed compilation errors when building without SSL.
6738c3a3b0 change: should match the local address when get connection from the keepalive pool.
7b6fa21abc feature: implemented keepalive pooling in 'balancer_by_lua*'.
4f8b943759 bugfix:main thread access free fake request.
892b7ee5e8 bugfix: lua-nginx-module context was clear when ngx.send_header() trigger filter_finalize case.
b5d1688ae7 bugfix: fix config test for signalfd with gcc 11.
5bf876104c doc: update doc for 'ngx.req.http_version'.
c1d309284d bugfix: worker thread Lua VM may take lots of memory.
6c00bd4765 bugfix: the connection won't be closed normally when set arg[1] = "" before arg[2] = true.
e2067ddd2b bugfix: wrong arguments of setkeepalive() result in the compromise of data integrity.
f725c60ea0 bugfix: Fixing compatibility issues with BoringSSL.
76e3d67a04 feature: validate and expose nextUpdate field in OCSP response.
0189eb14f6 feature: add suport for deriving key from tls master secret.
ca942b6984 feature: add udp cosocket bind api.
e5248aa820 bugfix: fixed HTTP HEAD request smuggling issue.
6394debe28 optimize: allow to reenable the tls for the upstream.
1654cc6a4d feature: add FFI function for balancer.disable_ssl.
6df2b6f01c bugfix: correct offset vector memory allocation size for PCRE2.
9e59105fd9 tests: fixed typo.
54e5cb09d3 feature: add ngx_http_lua_ffi_ssl_client_random.
1c77f025fa bugfix: fix memory corruption in consecutive regex calls.
8dec675832 docs: docs about ngx.read_body() API limitations in stream HTTP2 and HTTP3 processing case
6e29c1a96e Revert "bugfix: disable http2 in body read due to http2 stream processing bug."
e0d19f787e Revert "changes: modify read body api limitation for HTTP/2 or HTTP/3 requests."
7598ff389e feature: add ngx_http_lua_ffi_parse_der_cert and ngx_http_lua_ffi_parse_der_key functions.
git-subtree-dir: src/deps/src/lua-nginx-module
git-subtree-split: ea09d92adf835e30cab8d79343c7b8266e888128
33d6b04681 Merge branch 'v2.1' into v2.1-agentzh
ae4735f621 Reflect override of INSTALL_LJLIBD in package.path.
6f834087d0 ARM64: Use movi to materialize FP constants.
2d54213e7c Add more FOLD rules for integer conversions.
833600390c Merge branch 'master' into v2.1
86e7123bb1 Different fix for partial snapshot restore due to stack overflow.
7369eff67d Fix IR_ABC hoisting.
3bdc6498c4 Limit CSE for IR_CARG to fix loop optimizations.
a07ece864e Merge branch 'v2.1' into v2.1-agentzh
04dca7911e Call math.randomseed() without arguments to seed from system entropy.
7421a1b33c Restore state when recording __concat metamethod throws an error.
510f88d468 Add build flag LUAJIT_DISABLE_TAILCALL to disable tailcall generation.
444c8ff19a Clarify that lj_buf_shrink() does not keep any buffer data.
747fc02eb9 OSX: Fix installed luajit.pc.
8038430110 Merge branch 'master' into v2.1
7a608e4425 FFI: Fix various issues in recff_cdata_arith.
f602f0154b Fix predict_next() in parser (for real now).
f2a1cd4328 FFI: Fix __tostring metamethod access to enum cdata value.
0f8b878e2f Fix typo.
6885efb73e Merge branch 'master' into v2.1
811c5322c8 Handle partial snapshot restore due to stack overflow.
git-subtree-dir: src/deps/src/luajit
git-subtree-split: 33d6b04681d2f079a6d013988a426a841c52e29e
