Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-10 18:48:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
2614 commits 388 branches 195 tags 269 MiB
Commit graph

57 commits

Author SHA1 Message Date
Bernhard Amann
3559a39d59 introduce reader-info struct analogous to writer-info. 
All tests still pass.
 2012-07-02 10:03:28 -07:00
Bernhard Amann
f820ee9f5c Introduce support for a table of key/value pairs with further configuration options, 
with the same userinterface as in the logging interface.

Not really tested, but tests still work.
 2012-06-28 16:16:48 -07:00
Robin Sommer
41ef1072ec Changing an error in the input framework to a warning. 2012-06-15 16:25:31 -07:00
Robin Sommer
464732bfce Merge remote-tracking branch 'origin/topic/bernhard/input-threads-merge' 
* origin/topic/bernhard/input-threads-merge:
  disable streaming reads from executed commands.
  automatically delete disabled input streams
  small documentation fixes
  Documentation
 2012-06-15 15:39:58 -07:00
Bernhard Amann
852de4700c automatically delete disabled input streams 2012-06-07 14:12:42 -07:00
Jon Siwek
eeb1609768 Change Input::update_finished lookup to happen at init time. 
Also going through the internal_handler() function will set the
event as "used" (i.e. it's marked as being raised somewhere) and
fixes the core.check-unused-event-handlers test failure
(addresses #823).
 2012-05-31 15:32:28 -05:00
Robin Sommer
fc907c0090 A set of input framework refactoring, cleanup, and polishing. 2012-05-30 16:38:08 -07:00
Bernhard Amann
1416d5404d and another small memory leak when using streaming reads. 2012-05-29 10:35:56 -07:00
Bernhard Amann
f4864c69af fix another memory lead (when updating tables). 
Adjust twotables testcase - now it is faster. Shorten the output
-- because of threading, the results did not always come out in
the same order (it depends on which thread manages to sneak in
the results into the queue earlier).
 2012-05-29 09:21:16 -07:00
Robin Sommer
b37f9e38f6 Input framework merge in progress. 2012-05-25 16:16:53 -07:00
Bernhard Amann
658b188dff filters have been called streams for eternity. And I always was too 
lazy to change it everywhere...

Fix that.
 2012-05-25 16:11:30 -07:00
Bernhard Amann
2e452dc29f remove last remnants of autostart, which has been removed for quite a while. 2012-05-25 10:49:17 -07:00
Bernhard Amann
2034c10e97 make input framework source (hopefully) adhere to the usual indentation 
style. No functional changes.
 2012-05-25 10:33:22 -07:00
Bernhard Amann
82a6f3832a fix two memory leaks which occured when one used filters. 2012-05-22 13:51:50 -07:00
Bernhard Amann
3b82d69eb3 Merge remote-tracking branch 'origin/master' into topic/bernhard/input-threads 
Conflicts:
	src/CMakeLists.txt
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
 2012-05-18 15:26:36 -07:00
Bernhard Amann
1967f6f81c rename a couple of structures and make the names in manager fit the api more. 
This should it make easier for other people to understand what is going on without having knowledge of an "internal api * means * in external api" mapping.
 2012-04-15 15:11:39 -07:00
Bernhard Amann
a5cc98bb5d fix memory leak in tables and vectors that are read into tables 2012-04-10 13:57:09 -07:00
Bernhard Amann
8e526a7f83 fix memory leak for tables... nearly completely. 
There is still a tiny where I have not yet found where the delete could be missing.

For big table imports the memory footprint is significant nevertheless -- with tables of > 200000 entries, memory consumption can apparently reach in excess of 1.5Gb - and on a first glance this seems legitimate.

(The reason for this is probably that we use several hash tables to keep the performance impact small).
 2012-03-30 09:20:18 -07:00
Bernhard Amann
b7bbda7244 fix a couple more leaks. But - still leaking quite a lot with tables. 2012-03-30 09:20:12 -07:00
Bernhard Amann
7a71a74994 fix largest leak in manager. 2012-03-30 09:20:04 -07:00
Bernhard Amann
1170a87769 make benchmark reader hartbeat inverval aware 
fix small memleak on tablereader destruction
make timespread better configureable
 2012-03-30 09:19:01 -07:00
Bernhard Amann
9732859d44 add first simple benchmark reader (it simply spews random data, amount of lines specified in source). 2012-03-26 12:29:21 -07:00
Bernhard Amann
872ad195f7 prevent several remove operations for the same thread to be queued and output errors in that case. 2012-03-23 12:30:54 -07:00
Bernhard Amann
03116d779e one unref to many ... apparently 2012-03-22 18:08:59 -07:00
Bernhard Amann
94d439b0cb enable predicate modification of index of value which is currently being added/removed 
Todo: test if this works for removal ( I think it should ).
 2012-03-22 17:17:41 -07:00
Bernhard Amann
6c4a40f176 missing include on linux 2012-03-22 13:09:53 -07:00
Bernhard Amann
5f5209fcfb ...forgotten file. 2012-03-22 11:00:51 -07:00
Bernhard Amann
0db89bed3b fix crash when deleting data from source where there are no events or predicates... 
(that happens when all testcases are too complicated and use all features..)
 2012-03-22 10:33:49 -07:00
Bernhard Amann
51ddc9f572 fix bug that crashed input framework when creating already existing stream (tried to free not yet alloccated data) + write twotables test 2012-03-21 15:51:21 -07:00
Bernhard Amann
d39a389201 make optional fields possible for input framework. 
This do not have to be present in the input file and are marked as &optional in the record description.

Those can e.g. be used to create field values on the file in a predicate while reading a file - example:

	Input::add_table([$source="input.log", $name="input", $idx=Idx, $val=Val, $destination=servers,
				$pred(typ: Input::Event, left: Idx, right: Val) = { right$notb = !right$b; return T; }
 2012-03-20 14:11:59 -07:00
Bernhard Amann
aa6026c1a7 forgot to undo this - this idea did not work, because records cannot reference themselves. 2012-03-18 10:52:23 -07:00
Bernhard Amann
bf597012f8 fix some stupid, not that easy to find bugs. 
Functionality seems to work completely again - including all tests passing.
 2012-03-18 10:50:10 -07:00
Bernhard Amann
3286d013c9 forgot two files. 2012-03-16 23:45:10 -07:00
Bernhard Amann
e59aed6ce3 for seth - reverse order of event arguments 2012-03-16 08:31:19 -07:00
Bernhard Amann
842f635695 give EventDescripion field back to events 2012-03-16 08:10:28 -07:00
Bernhard Amann
367c4b4a7e make raw reading work. 
apparently there was a crash in the reader plugin, but main bro did not notice but waited for eternity for it do to something.
 2012-03-16 07:53:29 -07:00
Bernhard Amann
57ffe1be77 completely change interface again. 
compiles, not really tested.

basic test works 70% of the time, coredumps in the other 30 - but was not easy to debug on a first glance (most interestingly the crash happens in the logging framework - I wonder how that works).
Other tests are not adjusted to the new interface yet.
 2012-03-15 18:41:51 -07:00
Bernhard Amann
b4e6971aab Add regular debugging output for interesting operations (stream/filter operations) to input framework (this was way overdue) 2012-03-14 14:45:53 -07:00
Bernhard Amann
92555badd4 cleanup, more sanity tests, a little bit more documentation 2012-03-11 20:43:26 -07:00
Bernhard Amann
faf5c95752 a couple of small fixes ( default values, all null lines) 2012-03-11 19:41:41 -07:00
Bernhard Amann
7076c64a5e Merge remote-tracking branch 'origin/topic/robin/log-threads' into topic/bernhard/input-threads 
(and move a little bit of functionality from ascii reader to backend)

Conflicts:
	src/threading/Manager.cc
 2012-03-07 13:42:49 -08:00
Bernhard Amann
417542f283 Merge branch 'topic/bernhard/log-threads' into topic/bernhard/input-threads 
Seems to work -- all test pass.
But there are thread-safety issues at the moment, because the constructors of IPAddr and IPPrefix are not thread-safe, but needed by workers.

Conflicts:
	src/logging/Manager.cc
 2012-02-27 22:59:08 -08:00
Bernhard Amann
d553a3c6f6 fix strange bug when using predicates and events at the same time on a tablefilter. 
Testcase is now more involved.
 2012-02-23 15:30:39 -08:00
Bernhard Amann
d81607c3e9 fix empty field bug in threaded version 2012-02-23 14:36:04 -08:00
Bernhard Amann
93fac7a4be fix one of the bugs seth found in the input framework. 
(bug in PutTable when the table contained only one element and that element should not be wrapped into a record)
 2012-02-22 10:46:35 -08:00
Bernhard Amann
7e5f733826 raw input reader for seth, which can simply read a file into string-events given a line separator. 2012-02-22 09:44:45 -08:00
Bernhard Amann
d5b413c4e7 reduce number of needed hash operations 2012-02-20 17:13:41 -08:00
Bernhard Amann
edd30da082 better testcase & fix a few bugs (that took way too long to find). 2012-02-20 15:30:21 -08:00
Bernhard Amann
4f57817b1a ...forgotten in last commit. 2012-02-20 13:20:29 -08:00
Bernhard Amann
d21a450f36 add streaming reads & automatic re-reading of files to ascii reader. 
completely untested, but compiles & old tests still work
 2012-02-16 15:40:07 -08:00