This repository has been archived by the owner on Apr 1, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES
23887 lines (15107 loc) · 830 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
3.2.1 | 2020-09-08 14:48:20 -0700
* Release 3.2.1.
* Fix memory leak caused by re-entering AYIYA parsing (Jon Siwek, Corelight)
If the inner packet of a decapsulated AYIYA frame maps to the same
Connection as the outer packet and also contains another AYIYA
encapsulation, the same AYIYA analyzer attempts to parse the inner
packet. In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25256
* Fix memory leak caused by re-entering GTPv1 parsing (Jon Siwek, Corelight)
If the inner packet of a decapsulated GTPv1 frame maps to the same
Connection as the outer packet and also contains another GTPv1
encapsulation, the same GTPv1 analyzer attempts to parse the inner
packet. In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
* Exclude installing "zeek -> ." include dir symlink (Jon Siwek, Corelight)
The symlink only exists for use within the source-tree and isn't needed
for the install-tree.
* Fix build for PowerPC architecture (Johanna Amann, Corelight)
* Fix closing timestamp of rotated log files in supervised-cluster mode (Jon Siwek, Corelight)
* Fix issue with canonical names not printing for file analyzers (Tim Wojtulewicz, Corelight)
* GH-1120: Fix minimize_info in ftp/main not returning a value (Johanna Amann, Corelight)
3.2.0 | 2020-08-10 10:08:59 -0700
* Release 3.2.0.
3.2.0-rc1.4 | 2020-08-10 10:08:18 -0700
* GH-1100: Fix reported body-length of HTTP messages w/ sub-entities (Jon Siwek, Corelight)
The body-lengths of sub-entities, like multipart messages, got counted
twice by mistake: once upon the end of the sub-entity and then again
upon the end of the top-level entity that contains all sub-entities.
The size of just the top-level entity is the correct one to use.
3.2.0-rc1.3 | 2020-07-30 13:25:32 -0700
* Fix potential heap-use-after-free access (Dominik Charousset, Corelight)
3.2.0-rc1.2 | 2020-07-29 11:39:11 -0700
* GH-1083: Fix Input Framework 'change' events for 'set' destinations (Jon Siwek, Corelight)
3.2.0-rc1 | 2020-07-27 11:40:40 -0700
* Release 3.2.0-rc1.
3.2.0-dev.992 | 2020-07-27 11:36:53 -0700
* Fix potential NetbiosSSN analyzer stack overflow (Jon Siwek, Corelight)
The Contents_NetbiosSSN analyzer used a recursive message parsing
function that determined the size of the next message from the input
packet-data itself. A packet containing a sequence of many small
messages could cause a stack overflow since a recursion happened after
processing each message.
* Fix potential DNS analyzer stack overflow (Jon Siwek, Corelight)
The Contents_DNS analyzer used a recursive message parsing function that
determined the size of the next message from the input packet-data
itself. A packet containing a sequence of many small messages could
cause a stack overflow since a recursion happened after processing
each message.
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24272
3.2.0-dev.988 | 2020-07-27 11:19:20 -0700
* Update submodules to release versions in prep for release/3.2 branch (Jon Siwek, Corelight)
3.2.0-dev.986 | 2020-07-24 13:57:19 -0700
* Misc/minor cleanup after reviewing new Broker table store code (Jon Siwek, Corelight)
Just some naming/style + minor memory management tweaks (avoid
some copies; use move semantics).
3.2.0-dev.985 | 2020-07-24 11:40:22 -0700
* MySQL: Fix EOF parsing logic bug. (Vlad Grigorescu)
We were correctly NOT expecting an EOF, but because we were parsing the
header and then not parsing the rest, we would get out of sync
3.2.0-dev.979 | 2020-07-24 09:03:06 -0700
* Implement EDNS Client Subnet Option (Ron Wellman)
3.2.0-dev.974 | 2020-07-23 13:02:13 -0700
* origin/topic/jsiwek/gh-1076-fix-random:
Deprecate bro_srandom(), replace with zeek::seed_random().
Add zeek::max_random() & fix misuse of RAND_MAX w/ zeek::random_number()
Deprecate bro_random(), replace with zeek::random_number()
Deprecate bro_prng(), replace with zeek::prng()
GH-1076: Fix bro_srandom() to replace 0 seeds with 1
GH-1076: Fix bro_prng() implementation
GH-1076: Fix use of getrandom() (Tim Wojtulewicz, Corelight)
* Deprecate bro_srandom(), replace with zeek::seed_random().
Avoiding zeek::srandom() to avoid potential for confusion with srandom() (Jon Siwek, Corelight)
* Add zeek::max_random() & fix misuse of RAND_MAX w/ zeek::random_number()
In deterministic mode, RAND_MAX is not related to the result of
zeek::random_number() (formerly bro_random()), but some logic was
using RAND_MAX as indication of the possible range of values. The
new zeek::max_random() will give the correct upper-bound regardless
of whether deterministic-mode is used. (Jon Siwek, Corelight)
* Deprecate bro_random(), replace with zeek::random_number()
Avoiding the use of zeek::random() due to potential for confusion
with random(). (Jon Siwek, Corelight)
* Deprecate bro_prng(), replace with zeek::prng()
The type used for storing the state of the RNG is changed from
`unsigned int` to `long int` since the former has a minimal range
of [0, 65,535] while the RNG function itself has a range of
[1, 2147483646]. A `long int` must be capable of
[−2147483647, +2147483647] and is also the return type of `random()`,
which is what zeek::prng() aims to roughly parity. (Jon Siwek, Corelight)
* GH-1076: Fix bro_srandom() to replace 0 seeds with 1
The bro_prng() implementation cannot generate 0 as a result since it
causes every subsequent number from the PRNG to also be 0, so use the
number 1 instead of 0. (Jon Siwek, Corelight)
* GH-1076: Fix bro_prng() implementation
The intermediate result of the PRNG used unsigned storage, preventing
the ( result < 0 ) branch from ever being evaluated. This could cause
return values to exceed the modulus as well as RAND_MAX.
One interesting effect of this is potential for the rand() BIF to
return values outside the requested maximum limit.
Another interesting effect of this is that a PacketFilter may start
randomly dropping packets even if it was not configured for
random-packet-drops. (Jon Siwek, Corelight)
* GH-1076: Fix use of getrandom()
The availability and use of getrandom() actually caused unrandom and
deterministic results in terms of Zeek's random number generation. (Jon Siwek, Corelight)
3.2.0-dev.965 | 2020-07-23 08:31:41 -0700
* GH-1068: Add zeek symlink to allow "zeek/Foo.h" include style
Also changes `zeek-config --include_dir` to now output
`$prefix/include:$prefix/include/zeek` instead of only
`$prefix/include/zeek` (Jon Siwek, Corelight)
3.2.0-dev.962 | 2020-07-21 12:39:34 -0700
* Add zeek-archiver tool as submodule
This is added to the default Zeek build as a convenience since it's the
new suggested method for archiving the logs produced by a Supervised
Zeek Cluster. (Jon Siwek, Corelight)
3.2.0-dev.959 | 2020-07-21 15:34:59 +0000
* Broker Store table synchronizatio, (Johanna Amann, Corelight)
Zeek now supports synchronizing tables/sets across clusters using
a backing Broker store. The same feature also allows persistent
storage of data in tables/sets over Zeek restarts. This feature is
implemented using the new ``&backend`` attribute.
To synchronize a table over a cluster, you can, e.g., use:
global t: table[string] of count &backend=Broker::MEMORY;
This feature is documented in detail here:
https://docs.zeek.org/en/current/frameworks/broker.html#broker-store-backed-zeek-tables-for-data-synchronization-and-persistence
This feature is experimental and the syntax/specifics can change in the future.
3.2.0-dev.919 | 2020-07-17 16:37:11 -0700
* Use namespaced version of Location to silence warnings (Tim Wojtulewicz, Corelight)
3.2.0-dev.915 | 2020-07-17 16:10:46 -0700
* Various deprecation fixes, reported by failed Spicy builds (Tim Wojtulewicz, Corelight)
- Add deprecated version of ID::SetType() that takes Type*
- Add deprecated versions of zeek::set_location in the global namespace
- Fix global namespace version of lookup_ID to return ID*
* Add deprecated version of EnumType::GetVal() to return EnumVal* (Tim Wojtulewicz, Corelight)
Rename IntrusivePtr version to GetEnumVal
3.2.0-dev.911 | 2020-07-17 22:32:42 +0000
* Fix race condition in ensure_dir()
If something else created the dir between the stat() and mkdir(),
it previously reported that as a failure. (Jon Siwek, Corelight)
* Fix tokenize_string() to work with delimiters of length > 1. (Jon Siwek, Corelight)
3.2.0-dev.908 | 2020-07-17 10:19:07 +0000
* GH-734: Improve handling of lambdas that escape enclosing frame.
(Jon Siwek, Corelight)
3.2.0-dev.906 | 2020-07-17 10:08:31 +0000
* GH-837: Add test cases for mismatched Broker SSL configs. (Jon Siwek, Corelight)
* GH-837: Emit reporter errors for Broker errors. (Jon Siwek,
Corelight)
* Fix incorrect/missing Broker error status code numbers. (Jon Siwek, Corelight)
3.2.0-dev.902 | 2020-07-14 19:10:36 -0700
* GH-1062: fix integer conversion errors related to Tag subtypes (Jon Siwek, Corelight)
3.2.0-dev.899 | 2020-07-14 00:02:05 +0000
* Improve Broker store API's handling of invalid arguments
* Some methods mistakenly returned a bool instead of QueryResult
when passed an invalid `opaque of Broker::Store` handle.
* Now generates a runtime exception for store_name() and is_closed()
calls that pass an invalid `opaque of Broker::Store` handle as any
returned value can't be reasonably used in any subsequent logic.
* Descriptions of any invalid arguments are now given in the error
message. (Jon Siwek, Corelight)
* Add zeek::detail::emit_builtin_exception() functions
These work like zeek::emit_builtin_error(), but also throw an InterpreterException (Jon Siwek, Corelight)
* GH-1024: fix crash on passing wrong types to Broker store API (Jon Siwek, Corelight)
3.2.0-dev.894 | 2020-07-13 12:12:17 -0700
* GH-1019: deprecate icmp_conn params for ICMP events
Previously, a single `icmp_conn` record was built per ICMP "connection"
and re-used for all events generated from it. This may have been a
historical attempt at performance optimization, but:
* By default, Zeek does not load any scripts that handle ICMP events.
* The one script Zeek ships with that does handle ICMP events,
"detect-traceroute", is already noted as being disabled due to
potential performance problems of doing that kind of analysis.
* Re-use of the original `icmp_conn` record tends to misreport
TTL and length values since they come from original packet instead
of the current one.
* Even if we chose to still re-use `icmp_conn` records and just fill
in a new TTL and length value each packet, a user script could have
stored a reference to the record and not be expecting those values
to be changed out from underneath them.
Now, a new `icmp_info` record is created/populated in all ICMP events
and should be used instead of `icmp_conn`. It also removes the
orig_h/resp_h fields as those are redundant with what's already
available in the connection record. (Jon Siwek, Corelight)
3.2.0-dev.892 | 2020-07-13 12:10:22 -0700
* Fix wrong frame offsets for locals of alternate event/hook prototypes
Local frame offsets were being assigned based on number of the alternate
prototype's parameters, which may end up having less total parameters
than the canonical prototype, causing the local value to incorrectly
overwrite an event/hook argument value. (Jon Siwek, Corelight)
* Add deprecation expression to deprecated prototype/parameter messages (Jon Siwek, Corelight)
* Improve "use of deprecated prototype" warning message
The location information now points out the place of the deprecated
prototype instead of the location where the ID was initially declared
(which may not itself be a deprecated prototype). (Jon Siwek, Corelight)
* Emit deprecation warning for use of &deprecated function parameters
Particularly, this is meant for using &deprecated on canonical
event/hook prototype parameters to encourage users to create handlers
to another, non-deprecated prototype. i.e. for canonical prototypes,
we may not always want to put &deprecated directly on the prototype
itself since that signals deprecation of the ID entirely. (Jon Siwek, Corelight)
3.2.0-dev.885 | 2020-07-10 11:20:41 -0700
* Add more error checks to shadow log parsing (Jon Siwek, Corelight)
i.e. Coverity warns about possible use of ftell() negative return value
3.2.0-dev.884 | 2020-07-09 14:09:58 -0700
* Add Supervisor::{stdout,stderr}_hook (Jon Siwek, Corelight)
These allow capturing/handling the stdout/stderr of child processes
via Zeek scripts.
* Add Log::rotation_format_func and Log::default_rotation_dir options (Jon Siwek, Corelight)
These may be redefined to customize log rotation path prefixes,
including use of a directory. File extensions are still up to
individual log writers to add themselves during the actual rotation.
These new also allow for some simplication to the default
ASCII postprocessor function: it eliminates the need for it doing an
extra/awkward rename() operation that only changes the timestamp format.
This also teaches the supervisor framework to use these new options
to rotate ascii logs into a log-queue/ directory with a specific
file name format (intended for an external archiver process to
monitor separately).
* Allow Zeek function equality comparisons (Jon Siwek, Corelight)
Equality between two functions acts like a pointer comparison.
* Implement leftover log rotation/archival for supervised nodes (Jon Siwek, Corelight)
This helps prevent a node from being killed/crashing in the middle
of writing a log, restarting, and eventually clobbering that log
file that never underwent the rotation/archival process.
The old `archive-log` and `post-terminate` scripts as used by
ZeekControl previously implemented this behavior, but the new logic is
entirely in the ASCII writer. It uses ".shadow" log files stored
alongside the real log to help detect such scenarios and rotate them
correctly upon the next startup of the Zeek process.
* Move some supervisor structures to detail namespace (Jon Siwek, Corelight)
* StemHandle -> detail::SupervisorStemHandle
* SupervisedNode -> detail::SupervisedNode
* Supervisor::Node -> detail::SupervisorNode
* Improve default supervisor stdout/stderr handling (Jon Siwek, Corelight)
The stdout/stderr of child processes is now redirected over a pipe back
to the supervisor process so that it can prefix the output with
the name of the emitting node.
* Factor Supervisor initialization data to separate struct from Stem (Jon Siwek, Corelight)
* Move ParentProcessCheckTimer to detail namespace (Jon Siwek, Corelight)
3.2.0-dev.874 | 2020-07-08 17:46:20 -0700
* scan.l: Remove "constant" did_module_restore logic / dead code (Arne Welzel)
3.2.0-dev.872 | 2020-07-08 12:25:11 -0700
* Fix FreeBSD CI script to install right SWIG package (Jon Siwek, Corelight)
3.2.0-dev.870 | 2020-07-08 15:41:15 +0000
* Fix WhileStmt to call Stmt(Tag) constructor. (Jon Siwek,
Corelight)
3.2.0-dev.868 | 2020-07-08 15:39:46 +0000
* GH-928: use realpath() instead of inode to de-duplicate scripts.
(Jon Siwek, Corelight)
3.2.0-dev.864 | 2020-07-06 21:06:41 -0700
* GH-1040: Add zero-indexed version of str_split (Tim Wojtulewicz, Corelight)
3.2.0-dev.862 | 2020-07-06 20:40:44 -0700
* GH-1041: Move compress_path to a bif that uses normalize_path (Tim Wojtulewicz, Corelight)
3.2.0-dev.859 | 2020-07-06 14:55:00 +0000
* Add backtrace() and print_backtrace() BIF functions. (Jon Siwek,
Corelight)
3.2.0-dev.855 | 2020-07-03 14:29:55 -0700
* Fix --enable-mobile-ipv6 build (Jon Siwek, Corelight)
3.2.0-dev.854 | 2020-07-03 14:17:42 -0700
* Fix namespace of GetCurrentLocation() to zeek::detail (Jon Siwek, Corelight)
3.2.0-dev.853 | 2020-07-02 19:24:09 -0700
* Rename BroString files to ZeekString (Tim Wojtulewicz, Corelight)
* Rename BroObj to Obj (Tim Wojtulewicz, Corelight)
* Rename BroString to zeek::String (Tim Wojtulewicz, Corelight)
* Move Func up to zeek namespace, rename BroFunc to ScriptFunc (Tim Wojtulewicz, Corelight)
* Mark global val_mgr as deprecated and fix uses of it to use namespaced version (Tim Wojtulewicz, Corelight)
* Update binpac and bifcl submodules with review changes (Tim Wojtulewicz, Corelight)
* Move Location to zeek::detail and BroObj to zeek (Tim Wojtulewicz, Corelight)
* Move BroString to zeek namespace (Tim Wojtulewicz, Corelight)
* Move Dictionary/PDict, List/PList, and Queue/PQueue to zeek namespace (Tim Wojtulewicz, Corelight)
* Remove typedef that should have been removed in 3.1 (Tim Wojtulewicz, Corelight)
* Move Func and associated classes into zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Move Frame and Scope to zeek::detail namespace (Tim Wojtulewicz)
* Move all Val classes to the zeek namespaces (Tim Wojtulewicz, Corelight)
* Use type aliases for IntrusivePtr definitions (Tim Wojtulewicz, Corelight)
* Move deprecation macro to zeek-config.h.in to avoid having to over-include util.h (Tim Wojtulewicz, Corelight)
* Move IntrusivePtr and utility methods to the zeek namespace (Tim Wojtulewicz, Corelight)
3.2.0-dev.834 | 2020-07-02 15:57:09 -0700
* GH-786: fix Val::As methods casting incomplete types (Jon Siwek, Corelight)
* GH-786: fix Stmt::As methods casting incomplete types (Jon Siwek, Corelight)
* GH-786: fix Expr::As methods casting incomplete types (Jon Siwek, Corelight)
3.2.0-dev.830 | 2020-07-02 11:36:28 -0700
* GH-786: Move Type::As methods to cc file so they have type info for casting safely (Tim Wojtulewicz, Corelight)
3.2.0-dev.828 | 2020-07-01 09:57:23 -0700
* Fix a deprecation warning compiling with GCC (Jon Siwek, Corelight)
3.2.0-dev.827 | 2020-06-30 16:54:22 -0700
* Deprecate plugin::HookType and plugin::component::Type in a different way (Tim Wojtulewicz, Corelight)
* Deprecate init_class and IDScope in another way. (Tim Wojtulewicz, Corelight)
This also renamed init_class to InitClass for consistency.
* Deprecate TypeTag and friends in a different way (Tim Wojtulewicz, Corelight)
* Deprecate attr_tag in a different way, rename to AttrTag (Tim Wojtulewicz, Corelight)
3.2.0-dev.822 | 2020-06-30 15:54:38 -0700
* Revert Attributes::Attrs back to return an attr_list and mark it deprecated (Tim Wojtulewicz, Corelight)
* GH-1034: Revert TypeList::Types() back to return a type_list* and mark it deprecated (Tim Wojtulewicz, Corelight)
3.2.0-dev.817 | 2020-06-29 19:23:21 +0000
* GH-1036: change print.log to log network time instead of current (Jon Siwek, Corelight)
3.2.0-dev.815 | 2020-06-29 10:25:05 -0700
* reduce memory usage of ConnPolling (Justin Azoff)
Instead of scheduling the event with the full 'connection' record,
schedule it with the smaller 'conn_id' record.
3.2.0-dev.813 | 2020-06-26 16:25:34 +0000
* Fix several issues with command-line option redefs
* Variables of `string` type can now be set to an empty string
* Trying to set a variable with non-`string` type to an empty value
now emits an error instead of silently doing nothing
* Providing an invalid identifier now emits an "unknown identifier"
error instead of silently doing nothing (Jon Siwek, Corelight)
* Fix "possibly-truncated" compiler warning in BuildJSON snprintf() (Jon Siwek, Corelight)
* GH-1025: allow copying/cloning of `opaque of Broker::Store`
Implemented simply as a reference count increment of the
data store handle. (Jon Siwek, Corelight)
* Fix shadowed variable that breaks lookup_hostname(). (Jon Siwek, Corelight)
3.2.0-dev.804 | 2020-06-25 23:58:59 -0700
* Extend dns_request, dns_reject, and dns_query_reply events with original_query param (Ryan Victory)
Where the "query" parameter is downcased, the "original_query" is
passed unmodified to these events.
Added a policy/protocols/dns/log-original-query-case.zeek script that adds
the original_query to the dns.log file as well.
* Convert to using permissions to check for access to cirrus variables in benchmark script (Tim Wojtulewicz, Corelight)
3.2.0-dev.798 | 2020-06-19 13:25:02 -0700
* Remove last_access_time from TableEntryVal. (Johanna Amann, Corelight)
Turns out - this was no longer used. And it takes up at least 8 bytes
of space for every single table/set entry.
3.2.0-dev.796 | 2020-06-18 20:40:59 +0000
* cmake: Make musl support more distro agnostic (Andrew Benson)
3.2.0-dev.794 | 2020-06-18 20:04:06 +0000
* Minimize data published for expected FTP data channel analysis
Previously, more data than could effectively be utilized by any remote
Zeek was published (e.g. full list of pending commands or other
transient state that may add up to non-trivial amount of bytes). (Jon Siwek, Corelight)
* GH-998: Fix Reporter::conn_weird() to handle expired connections
This introduces a new sampling state-map for expired connections to fix
segfaults that previously occured when passing in a `connection` record
to `Reporter::conn_weird()` for which the internal `Connection` object
had already been expired and deleted. This also introduces a new event
called `expired_conn_weird`, which is similar to `conn_weird`, except
the full `connection` record is no longer available, just the `conn_id`
and UID string. (Jon Siwek, Corelight)
* Place build file in explicit location for benchmarking to work correctly (Tim Wojtulewicz, Corelight)
* Update highwayhash submodule to upstream.
This includes a patch by us, which fixes the compile on AARCH64.
Fixes GH-1006. (Johanna Amann, Corelight)
3.2.0-dev.788 | 2020-06-15 11:04:20 -0700
* Fix location where CI places build.tgz (Jon Siwek, Corelight)
3.2.0-dev.787 | 2020-06-15 10:27:48 -0700
* Add new CI task for running benchmarks on a remote host (Tim Wojtulewicz, Corelight)
3.2.0-dev.783 | 2020-06-11 23:21:41 -0700
* Compare pcap_next_ex() result to PCAP_ERROR/PCAP_ERROR_BREAK (Jon Siwek, Corelight)
* GH-977: Improve pcap error handling (Jon Siwek, Corelight)
Switches from pcap_next() to pcap_next_ex() to better handle all error
conditions. This allows, for example, to have a non-zero exit code for
a Zeek process that fails to fully process all packets in a pcap file.
* Remove not-useful code in iosource::Manager::OpenPktSrc (Jon Siwek, Corelight)
3.2.0-dev.779 | 2020-06-11 23:17:46 -0700
* Rename BroType to zeek::Type (Tim Wojtulewicz, Corelight)
* Move Type types to zeek namespace (Tim Wojtulewicz, Corelight)
* Move Flare/Pipe from the bro namespace to zeek::detail (Tim Wojtulewicz, Corelight)
* Move Attr to the zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Move Trigger into the zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Move ID to the zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Move Anon.h into zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Mark all of the aliased classes in plugin/Plugin.h deprecated (Tim Wojtulewicz, Corelight)
And fix all of the plugins that were using them
* Move all of the base plugin classes into the zeek::plugin namespace (Tim Wojtulewicz, Corelight)
* Expr: move all classes into zeek::detail (Tim Wojtulewicz, Corelight)
* Stmt: move Stmt classes into zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Add utility macro for creating namespaced aliases for classes (Tim Wojtulewicz, Corelight)
3.2.0-dev.763 | 2020-06-10 16:34:31 -0700
* Optimize Connection::RemovalEvent() for bare-mode usage (Jon Siwek, Corelight)
It was creating RecordVals even if they wouldn't be used by any event
handler and that situation is common/expected for `zeek -b` mode.
Normally, there's at least the tunnel scripts with a `new_connection`
handler causing the connection RecordVals to be built.
3.2.0-dev.762 | 2020-06-09 15:19:15 -0700
* Restrict Cirrus CI to only zeek repo's branches (Jon Siwek, Corelight)
3.2.0-dev.761 | 2020-06-09 09:39:31 -0700
* Prevent IP fragment reassembly on packets without minimal IP header (Jon Siwek, Corelight)
The IP fragment reassembly process assumes a packet contains at least
the minimum IP header, but such a check did not previously occur,
resulting in a heap buffer over-read. For example, a self-reported
IPv4 IHL field with a value less than minimum IPv4 header length of
20 bytes. Such packets likely aren't routable on their own, but one
can create an artifical pcap like that or possibly encapsulate it
within another protocol to trigger this bug.
* Fix potential leak of Analyzers added to tree during Analyzer::Done (Jon Siwek, Corelight)
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22630
* Fix multiple content-transfer-encoding headers causing a memory leak (Jon Siwek, Corelight)
The MIME analyzer may also leak if it sees many multipart boundary
parameters.
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22871
* Fix potential stack overflow in NVT analyzer (Jon Siwek, Corelight)
The NVT_Analyzer (e.g. as instantiated to support the FTP analyzer)
uses a recursive parsing function that may only advance one byte at a
time and can easily cause a stack overflow as a result. This change
replaces the recursive calls with equivalent iterative logic.
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22898
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22972
* Fix NVT analyzer memory leak from multiple telnet authn name options (Jon Siwek, Corelight)
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=23069
3.2.0-dev.751 | 2020-06-08 22:30:18 +0000
* GH-999: Stop formatting DHCP Client ID Hardware Type 0 as MAC
For `DHCP::ClientID$hwtype` fields equal to 0, the `hwaddr` field is
no longer misformatted as a MAC and instead just contains the raw bytes
seen in the DHCP Client ID Option. (Jon Siwek, Corelight)
3.2.0-dev.749 | 2020-06-08 11:22:45 -0700
* Remove inline from some static KeyedHash members (Jon Siwek, Corelight)
Coverity Scan builds currently encounter catastrophic error, claiming
alignas requires use on both declaration and definition, so appears to
actually not understand "static inline" in combo with alignas.
* Improve Func.h inclusion (Jon Siwek, Corelight)
Now forward declares some Broker types since Broker/CAF headers
generally slow things down and also Coverity Scan currently has a
catastrophic error on some CAF headers.
Also a few other changes to EventHandler/BifReturnVal to reduce number
of places that depend on Func.h.
* Rename aux/ to auxil/ (Jon Siwek, Corelight)
Since "aux" is not an allowed file/dir name on Windows.
3.2.0-dev.744 | 2020-06-04 15:11:56 -0700
* Fix use-after-move of proc_status_file breaking -U flag (Jon Siwek, Corelight)
* Change Cirrus CI config to use org-level secured variable (Jon Siwek, Corelight)
3.2.0-dev.742 | 2020-06-03 13:29:12 -0700
* GH-973: Fix the return type of topk_get_top()
e.g. when using `string` elements it should return `vector of string`,
not `vector of list of string` like it did before. (Jon Siwek, Corelight)
3.2.0-dev.740 | 2020-06-03 13:17:26 -0700
* Deprecate Val(double, TypeTag) ctor, add TimeVal/DoubleVal subclasses
This also updates all usages of the deprecated Val ctor to use
either IntervalVal, TimeVal, or DoubleVal ctors. The reason for
doing away with the old constructor is that using it with TYPE_INTERVAL
isn't strictly correct since there exists a more specific subclass,
IntervalVal, with overriden ValDescribe() method that ought to be used
to print such values in a more descriptive way. (Jon Siwek, Corelight)
3.2.0-dev.738 | 2020-06-02 18:13:50 -0700
* GH-985: Fix descriptions of double_to_interval() return values (Jon Siwek, Corelight)
The BIF was not returning an IntervalVal which has an overriden
ValDescribe() method that allows for prettier printing like "6.0 secs"
instead of just "6.0".
3.2.0-dev.736 | 2020-06-02 12:37:56 -0700
* Decrease number of CPUs/memory for Cirrus CI tasks (Jon Siwek, Corelight)
3.2.0-dev.735 | 2020-06-02 10:55:43 -0700
* Change CI script to compile from build/Makefile (Jon Siwek, Corelight)
3.2.0-dev.734 | 2020-06-01 20:44:29 -0700
* Add missing include in util.cc (Jon Siwek, Corelight)
3.2.0-dev.733 | 2020-06-01 19:25:37 -0700
* Remove Analyzer.h from bro-bif.h (Tim Wojtulewicz, Corelight)
* Remove IPAddr.h from Reporter.h (Tim Wojtulewicz, Corelight)
* Remove the inclusion of Func.h from NetVar.h (Tim Wojtulewicz, Corelight)
3.2.0-dev.727 | 2020-06-01 10:57:16 -0700
* Integrate review feedback
* Add deprecation for MIME_Entity::ContentType(), use GetContentType()
* Add deprecation for MIME_Entity::ContentSubType(), use GetContentSubType()
* Add deprecation for MIME_Message::BuildHeaderVal(), use ToHeaderVal()
* Add deprecation for MIME_Message::BuildHeaderTable(), use ToHeaderTable()
* Add deprecation for mime::new_string_val(), use mime::to_stringval()
* Add deprecation for ARP_Analyzer::ConstructAddrVal(), use ToAddrVal()
* Add deprecation for ARP_Analyzer::EthAddrToStr(), use ToEthAddrStr()
* Change the Func::Call() replacement to be named Func::Invoke() (Jon Siwek, Corelight)
* Switch Broker Val converter visitor to return IntrusivePtr (Jon Siwek, Corelight)
* Change BroFunc ctor to take const-ref IntrusivePtr<ID> (Jon Siwek, Corelight)
* Add version of Frame::SetElement() taking IntrusivePtr<ID>
Expect the version using raw ID* could go away eventually, but this is
convenience for the meantime. (Jon Siwek, Corelight)
* Change Scope/Func inits from id_list* to vector<IntrusivePtr<ID>> (Jon Siwek, Corelight)
* Change Scope::GenerateTemporary() to return IntrusivePtr (Jon Siwek, Corelight)
* Deprecate Scope::ReturnType(), replace with GetReturnType() (Jon Siwek, Corelight)
* Deprecate Scope::ScopeID(), replace with GetID() (Jon Siwek, Corelight)
* Switch parsing to use vector<IntrusivePtr<Attr>> from attr_list
This allows improved passing/storing of Attr references to Exprs,
TypeDecl, Scope, etc. (Jon Siwek, Corelight)
* Deprecate TableVal::FindAttr(), replace with GetAttr() (Jon Siwek, Corelight)
* Deprecate TypeDecl::FindAttr(), replace with GetAttr() (Jon Siwek, Corelight)
* Deprecate ID::FindAttr(), replace with GetAttr() (Jon Siwek, Corelight)
* Deprecate Attributes::FindAttr(), replace with Find() (Jon Siwek, Corelight)
* Deprecate Attributes::AddAttrs(Attributes*)
Replaced with version taking an IntrusivePtr parameter (Jon Siwek, Corelight)
* Add Attributes ctor that takes IntrusivePtrs (Jon Siwek, Corelight)
* Change Attributes to store std:vector<IntrusivePtr<Attr>>
This also changes the return type of Attributes::Attrs() from attr_list* (Jon Siwek, Corelight)
* Change Attr::SetAttrExpr() to non-template (Jon Siwek, Corelight)
* Deprecate Attr::AttrExpr(), replace with GetExpr() (Jon Siwek, Corelight)
* Deprecate ID::Attrs(), replace with GetAttrs() (Jon Siwek, Corelight)
* Remove weak_ref param from ID::SetVal()
It was not used anywhere. (Jon Siwek, Corelight)
* Store weak ref boolean along with Frame element Vals (Jon Siwek, Corelight)
* Deprecate Frame::GetElement(ID*), replace with GetElementByID() (Jon Siwek, Corelight)
* Deprecate Frame::NthElement(), replace with GetElement() (Jon Siwek, Corelight)
* Switch Frame::SetElement() to use IntrusivePtr (Jon Siwek, Corelight)
* Store IntrusivePtrs in Frame (Jon Siwek, Corelight)
* Deprecate Plugin::HookCallFunction(), replace with HookFunctionCall()
This also changes the argument type of Func::operator() to zeek::Args*
to allow plugins to be able to alter function arguments in place as
was previously documented. (Jon Siwek, Corelight)
* Switch plugin::Manager::HookCallFunction() to return IntrusivePtr
The plugin::Plugin side of things is not (yet) changed. (Jon Siwek, Corelight)
* Port remaining file analysis API to use IntrusivePtr (Jon Siwek, Corelight)
* Deprecate file analyzer construction methods taking raw RecordVal*
Replaced with versions that instead take IntrusivePtr (Jon Siwek, Corelight)
* Deprecate file_analysis::Analyzer::Args(), replace with GetArgs() (Jon Siwek, Corelight)
* Deprecate file_analysis::File::GetVal(), replace with ToVal() (Jon Siwek, Corelight)
* Change file_analysis::File::GetID() to return const-ref (Jon Siwek, Corelight)
* Fix build: some don't like IntrusivePtr default arg w/ incomplete type (Jon Siwek, Corelight)
* Change bro_broker::DataVal::ScriptDataType() to IntrusivePtr (Jon Siwek, Corelight)
* Change find_nested_record_types() to take IntrusivePtr (Jon Siwek, Corelight)
* Deprecate TypeType::Type(), replace with GetType() (Jon Siwek, Corelight)
* Add is_assignable() overload taking TypeTag (Jon Siwek, Corelight)
* Add is_atomic_type() overloads for IntrusivePtr (Jon Siwek, Corelight)
* Add same_type() overloads for IntrusivePtr args (Jon Siwek, Corelight)
* Change merge_types() to take IntrusivePtrs (Jon Siwek, Corelight)
* Deprecate IndexType::Indices(), replace with GetIndices() (Jon Siwek, Corelight)
* Add is_vector() methods taking const-ref IntrusivePtr (Jon Siwek, Corelight)
* Factor static-local nil IntrusivePtrs to global locations
Minor optimization to remove any run-time impact. (Jon Siwek, Corelight)
* Deprecate VectorVal::Lookup(), replace with At() (Jon Siwek, Corelight)
* Switch VectorVal BroValUnion to store std::vector<IntrusivePtr<Val>>
This changes the return type of AsVector() from std::vector<Val*>* (Jon Siwek, Corelight)
* Deprecate VectorVal::Insert() taking raw Val*, use IntrusivePtr (Jon Siwek, Corelight)
* Deprecate VectorVal::Assign methods taking raw Val*
And adapt usages to pass in to alternate method taking IntrusivePtr (Jon Siwek, Corelight)
* Merge remote-tracking branch 'origin/master' into topic/jsiwek/gh-893-intrusive-ptr-migration (Jon Siwek, Corelight)
* Change EventHandler to store IntrusivePtr<Func>
Also deprecates the LocalHandler() and SetLocalHandler() methods,
replaced with GetFunc() and SetFunc(). (Jon Siwek, Corelight)
* Add Val::AsFuncPtr() convenience method
Since it's not trivial to store IntrusivePtr in BroValUnion and also
not satisfying to store IntrusivePtr*. (Jon Siwek, Corelight)
* Deprecate StringVal::Substitute(), replace with Replace() (Jon Siwek, Corelight)
* Switch a TableVal::CallChangeFunc param to IntrusivePtr (Jon Siwek, Corelight)
* Fix ambiguous ODesc::Add() call (Jon Siwek, Corelight)
* Minor TableVal::Assign() ref-counting optimization (Jon Siwek, Corelight)
* Improve TableVal HashKey management
* Deprecated ComputeHash() methods and replaced with MakeHashKey()
which returns std::unique_ptr<HashKey>
* Deprecated RecoverIndex() and replaced with RecreateIndex()
which takes HashKey& and returns IntrusivePtr.
* Updated the new TableVal Assign()/Remove() methods to take either
std::unique_ptr<HashKey> or HashKey& as appropriate for clarity of
ownership expectations. (Jon Siwek, Corelight)
* Add back in a deprecated TableVal ctor taking raw pointers (Jon Siwek, Corelight)
* Deprecate TableVal::Attrs(), replace with GetAttrs() (Jon Siwek, Corelight)
* Deprecate TableVal::Delete(), replace with Remove() (Jon Siwek, Corelight)
* Switch Func::Call(val_list*) back to returning Val*
And renamed the method returning IntrusivePtr to operator().
This corrects the deprecation process for Func::Call(val_list*). (Jon Siwek, Corelight)
* Deprecate TableVal::Lookup(), replace with Find()/FindOrDefault() (Jon Siwek, Corelight)
* Deprecate TableVal IsSubsetOf and EqualTo taking Val*, use Val& (Jon Siwek, Corelight)
* Deprecate ComputeHash(Val*) methods, replace with ComputeHash(Val&) (Jon Siwek, Corelight)
* Deprecate TableVal::Intersect(), replace with Intersection() (Jon Siwek, Corelight)
* Deprecate TableVal::Assign methods with Val*, add IntrusivePtr overloads (Jon Siwek, Corelight)
* Deprecate RecordVal::Lookup(const char*, bool)
Replace with GetField(const char*) and GetFieldOrDefault(const char*). (Jon Siwek, Corelight)
* Deprecate RecordVal::LookupWithDefault() replace with GetFieldOrDefault()
(The former was previously changed during this release cycle to return
Intrusive pointer, but this just changes it back to return Val* and
deprecates it). (Jon Siwek, Corelight)
* Deprecate RecordVal::Lookup(int), replace with GetField(int) (Jon Siwek, Corelight)
* Change BroValUnion to use IntrusivePtr for record field storage
This also changes the AsRecord() and AsNonConstRecord() accessors
to return std::vector<IntrusivePtr<Val>>* instead of val_list* (Jon Siwek, Corelight)
* Switch ASN1 Val conversion functions to return IntrusivePtr (Jon Siwek, Corelight)
* Deprecate RecordVal::Assign(int, Val*)
And adapt all usages to the existing overload taking IntrusivePtr. (Jon Siwek, Corelight)
* Switch RPC analyzers to use IntrusivePtr (Jon Siwek, Corelight)
* Switch RecordVal::CoerceTo() to use IntrusivePtr (Jon Siwek, Corelight)
* Deprecate TableEntryVal::Value(), replace with GetVal() (Jon Siwek, Corelight)
* Switch OpaqueVal::SerializeType() to IntrusivePtr (Jon Siwek, Corelight)
* Switch BlommFilterVal/CardinalityVal to use IntrusivePtr<BroType> (Jon Siwek, Corelight)
* Switch some TopkVal methods to use IntrusivePtr (Jon Siwek, Corelight)
* Switch TopkVal to store IntrusivePtr<BroType> (Jon Siwek, Corelight)
* Switch OpaqueVal::UnserializeType() to return IntrusivePtr (Jon Siwek, Corelight)
* Deprecate FuncType::ArgTypes(), replace with ParamList() (Jon Siwek, Corelight)
* Deprecate FuncType::Args(), replace with Params() (Jon Siwek, Corelight)
* Deprecate EventHandler::FType(), replace with GetType() (Jon Siwek, Corelight)
* Deprecate BroFile::FType(), replace with GetType() (Jon Siwek, Corelight)
* Deprecate Func::FType(), replace with Func::GetType() (Jon Siwek, Corelight)
* Change {Get,Set}ContentsFile() to use IntrusivePtr (Jon Siwek, Corelight)
* Deprecate BroFile::GetFile(), replace with BroFile::Get() (Jon Siwek, Corelight)
* Deprecate Val(BroFile*) ctor, replace with one using IntrusivePtr (Jon Siwek, Corelight)
* Deprecate Val(Func*) ctor, replace with one using IntrusivePtr (Jon Siwek, Corelight)
* Deprecate OpaqueVal/HashVal ctors that take OpaqueType*
Replaced with ones that take IntrusivePtr (Jon Siwek, Corelight)