VoIP monitor Files

7.0.1
=====

- fix capture rules direction logic. It was always only by destinatino instead of "both". Broken since version 7


6.5.4 -> 7.0 10.6.2013
======================

New features
------------
- implement Cisco SKINNY (SCCP) VoIP protocol (still beta) 
- use MySQL partition feature by default. 
- implement filter by SIP type into live sniffer
- redesign realtime audio listening (g711 ulaw/alaw only at the moment) 
- enable storing custom sip headers to database column cdr_next.custom_header_headername. You can specify more headers # delimited by ";". WARNING - when you enable this feature voipmonitor will autoupgrade cdr_next table which can take # hours depending on how large the table is. In GUI there is new section Settings#Custom_headers. WARNING - when you enable this feature voipmonitor will autoupgrade cdr_next table which can take hours depending on how large the table is INFO: in GUI there is new section in Settings - http://www.voipmonitor.org/doc/Settings#Custom_headers
- print backtrace to syslog in case voipmonitor crashes
- detect duplicate packets (if enabled) even if they aren't the "next packet". Patch provided by Steve Davies <steve@connection-telecom.com>
- implement storing DTMF to new cdr_dtmf table for RFC2833 and SIP INFO
- add convertchar option to voipmonitor.conf

Bug fixes
---------
- fix race condition which can lead to crash if ringbuffer is short and ip traffic is large and there are IP fragmented packets and ringbuffer runs faster then voipmonitor can handle
- fix TCP SIP reassemble logic which were mixing packets between pcap files in certain cases. 
- jitterbuffer f1 was to much sensitive reporting loss although delays was < 10ms
- fix low MOS score for calls which contains RTP DTMF (and fix audio sync too) 
- never count MOS for calls where packets are < 100 which also causes low MOS for no reason. 
- fix ip capture rules on 32 bit platform - all higher numbered IPs overlapped due to atoi limitations on 32bit
- fix "call not found" for listen manager command on 32 bit platforms. 
- create spool and cachedir directory upon start which fixes segfault in case cachedir does not exists
- if packet contains trailing header (VSS monitoring ethernet trailer for example) deduplication did not worked if one packet has it and the other not.
- do not save deduplicated packets in pcap file if this feature is enabled
- do not list calls in manager api which is hanged up or canceled or failed. There was always 5-30 second delay. 
- fix saving REGISTER pcap files in case cachedir is used
- fix missing RTP stream in CDR and short graph files (caler or called) in case sniffer see multiple RTP streams (more than 2) within call- treat REGISTER 403 response as failed too (not only 401)- if sniffer stores RTP packets but only headers (savertp = header) and there is rule set in capture rules to record full RTP - it was not recording full RTP packets- "libpcap or interface dropped some packets" counters are unsigned - fix showing negative numbers in syslog
- reformat help text to 80 chars and sort it alphabetically 


Changelog 6.5.3 -> 6.5.4
========================
- fix TCP manager interface which was limited to some maximum number of connections since start. As a result no new connection can be created and thus the WEB GUI locked. 


Changelog 6.5.2 -> 6.5.3
========================
- do not end manager thread if there is problem with client reading data. Ending manager thread causes PHP to lock whole session. 
- divide rtcp jitter by timestamp unit (8/12/16/24/32) so it will be in milliseconds to match RTP jitter
- don not endless try mysql connect when kill signal sends to voipmonitor. this caused stuck in reboot process of whole machine or impossible break voipmonitor ctrl-c


Changelog 6.5 -> 6.5.2
======================

- fix reversed RTP in mysql - caller RTP was called RTP (broken since 6.5RC1)
- fix 6.5 version 4GB memory leak for any manager command causing live calls to swap machine or crash depending on available memory

Complete changelog between 6.0.1 -> 6.5
=======================================

WARNING - it will auto upgrade your tables which can take hours when you first run the sniffer. 

New features
------------
- supports G722 audio decoding (it still needs G722 codec)
- Implement pausing RTP/WAV recording if DTMF sequence detected. New option pauserecordingdtmf
- implements on-demand live sniffing (to sql table)
- implement IP defragmentation for IP and IP in IP too
- support for SILK and iSAC codecs
- new option (cdronlyanswered = yes) write CDR to database only if call was answered
- new option (cdronlyrtp = yes) write CDR to database only if call has RTP
- reimplement ODBC database support for MSSQL
- support UTF8 for SIP MESSAGES
- try to rebind manager interface on specified port (default 5029) if the bind fails
- implement manager command disablecdr and enablecdr which will stop/start writing data to CDR.
- implement getversion manager command
- add content-type header to SIP MESSAGE sql table
- implements new cdr_rtp SQL table which holds all RTP streams which is used in WEB GUI sip history diagram
- live capture for SIP REGISTER 
- store SIP REGISTER to pcap based on capture rules defined in mysql table (WEB GUI)
- store id_sensor to register* tables
- implement --pcapscan-dir command line option

Optimizations
-------------
- optimize number of MYSQL commands for SIP REGISTER to be able to store via remote database where the network latency causes maximum number of queries.
- insert CDR in batch - increases CDR per second throughpat in case sensor is storing remotly
- new spooldir schema stores all files to year-mon-day/hour/minute/[ALL|SIP|RTP|AUDIO] directories if you need to have the old schema year-mon-day/* enable spooldiroldschema = yes.
- when store REGISTER into active table which is MEMORY type and mysql has enabled binlog (for replication) - send sql_log_bin = 0 before the command and sql_log_bin = 1 after the command to safe I/O
- tcp manager is now multithreaded which means that it can server multiple clients at once and thus not blocks other clients if some client connects to manager
- voipmonitor now by default splits SIP and RTP packets to individual files (in case spooldiroldschema = no) which are located in SIP and RTP directories. This feature allows instance cleaning RTP streams dif ferently then SIP packets. To join two pcap files SIP+RTP use mergecap command line utility which is included in wireshark package.
- 5 second tolerance for SIP REGISTER so it will not fill table with expired registrations when it register again after 1 second timeout

Bug fixes
---------
- update expire value from 200 OK replies to REGISTER 
- UserAgent string in database was reverted - caller was called and vice versa (probably since beginning..)
- fix audio sync in case there are more then 1 mark bit set in RTP. Some devices sends MARK bit since the beginning of RTP stream until opposite side sends RTP
- fix max and average jitter calculations which was broken since voipmonitor creation. Now jitter is the same as you can see in wireshark.
- fix audio in case ptime changes during call (for example iSAC changes between 30 and 60 dynamically)
- fix codec handling for ilbc which shows as G711 due to case sensitive searching in SDP
- fix one way call in some cases (where voipmonior listens in multi sip proxy environment and see all packets)
- fix 16khz codecs which resulted in bad graphs
- fix TCP reassemble
- fix crashes when using pcapscandir
- fix WAV for SILK 24khz
- created procedure __insert has to have unique name in case of multiple sensors (thanks Dmitry Sytchev)
- add / at the end of path for find command so it can find and clean directory which is also mounted. in other words autodelte do not work if the directory is mounted because of missing /
- fix mysql 5.6 not inserting queries if any of column value is out of range and the mysql server set strict mode which 5.6 is set by default
- when defragmenting UDP packets resets frag flag 

version 6.0.1 10.1.2013
=======================

- fix critical bug which cuased not closing calls in memory (racecondition in threads)


version 6.0 5.1.2013
====================

- Now the voipmonitor returns memory back to linux and not grow giantly in case of call peaks. This is because it allocates memory in a number of small chunks. When freeing one of those chunks, the OS may decide that giving this little memory back to the kernel will cause too much overhead and delay the operation. As all chunks are this small, they get actually freed but not returned to the kernel. On systems using glibc, there is a function call malloc_trim from malloc.h which does this missing operation (note that it is allowed to fail).
- Implement SIP message parsing and storing to new sql.message table
- fix vmbuffer size when set over 2000 it overlaps. now it is possible to set vmbuffer to 4000 check ringbuffer value which can be max 2000. if set to higher number do not overlap but limit to 2000
- fix memory corruption if voipmonitor uses rtp threads (which is by default) and in case the system gets slow due to swaping I/O blocking or CPU blocking


version 5.5 14.12.2012
======================

- fix cdrtable.sql 
- fix match_header in voipmonitor.conf - it is matchheader (support both of them) 
- fix memory corruptions and crashes in some more places when reading unexpected SIP messages. 
- fix crash when watching live calls via TCP manager (race condition in calls list)
- fix receiving ip in ip packets when reading live from interface
- fix compile on Suse (undeclared sleep)
- properly close TCP manager socket when ending voipmonitor
- implement mirroring SIP packets to specified IP address. IP is mirrored by ip in ip protocol. This is usefull in case the primary voipmonitor machine sees all SIP packets but only part of RTP packets and the second voipmonitor server see only RTP packets -> in this case you can mirror SIP packets to second voipmonitor server IP. voipmonitor already supports ip in ip so no configuration is needed to accept ip in ip mirrored packets. (this mirroring supports for example acme packet SBC).  mirroring can be also usefull for another general purpose - it is also parsed by wireshark/tshark etc.
- new reading packets option by scanning directory for pcap files. This is in conjuction with running tcpdump which creates pcap file each 5 seconds for example into /dev/shm ramdisk and voipmonitor reads file by file. For more details read comments in voipmonitor.conf or sniffer manual 5.5
- keep unreplied REGISTER messages in memory only for 30 seconds instead of rtptimeout


version 5.4.1 4.12.2012
=======================

- fix crash when reloading capture rules - move mysql functions into mysql thread to avoid race condition.
- fix register table creation
- do not mark T38 call if the latest SDP is not T.38 which means the T38 failed. But retain that flag for later use (will be implemented as flags in future)
- add sipcalledip to register_state and register_filter


version 5.4 30.11.2012
======================

- fix random memory corruption which leads to coredump when MAX_IP_PER_CALL was reached
- implement getwav genwav manager commands to allow wav download remotly
- name initial SIP last reposne = "NO RESPONSE" with code 0
- implement cleaning spool dir (set interval and size)
- add WARNING in voipmonitor.conf - pcapcommand is implemented by forking which is very expensive and is causing TLB shootouts on multicore system which can generate 500 000 interrupts / sec  causing system to drop packets. Watch the performance carefuly (with "vmstat 1" column "in"). Gziping pcap files will be implemented as native function directly in C++ to obey TLB shootdowns.
- do not overwrite pcap file if it exists
- search voipmonitor binary in sbin first then in bin
- mkdir /var/spool/voipmonitor/ in install script
- reset callend timer on each INVITE to not hangup call in case voipmonitor is not sniffing RTP 
- change default behaviour and do not save :port in domain column. To change it to old behaviour you can set domainport=yes in voipmonitor.conf
- add to_domain to register_state and register_failed tables and treat REGISTER unique for to_domain too instead of only by digestusername and to_num/to_domain
- fix cdr.called/caller when @ character is in sip name ["@home company"<sip:1234@domain>]


version 5.3.2 9.11.2012
=======================
- fix sip last response again - all 200 OK was accidentaly treated as answer to call so canceled calls instead of 487 code had 200 OK like answered calls


version 5.3.1 6.11.2012
=======================

- recognize fax calls also if rtpfirstleg is enabled
- fix lastsipresponse for connected calls which was 0 BYE instead of 200 OK


version 5.3 5.11.2012
=====================

- implement matchheader config in voipmonitor.conf which will save provided SIP header to cdr_next.match_header which is then used in WEB GUI to find all CDR legs. Typical is matchheader = in-reply-to. It is turned of by default and if enabled the database is automatically altered once the voipmonitor run which can take minutes - hours depends on number of rows in cdr_next and I/O speed.- fix cdrtable.sql.* typo for sensors table
- fix wav decode for GSM
- add rtptimeout (voipmonitor.conf), -m, --rtp-timeout it is important value which specifies how much seconds from the last SIP packet or RTP packet is call closed and writen to database. It means that if you need to monitor ONLY SIP you have to set this to at leat 2 hours = 7200 assuming your calls is not longer than 2 hours. Take in mind that seting this to very large value will cause to keep call in memory in case the call lost BYE and can consume all memory and slows down the sniffer - so do not set it to very high numbers. Default is 300 seconds. rtptimeout = 300
- remove custom_header1 from cdr table as it is not used (it is in cdr_next)- do not store last sip response to BYE but retain it for the invite.


version 5.2 29.10.2012
======================

- fixes some SIP REGISTER issues
- fix savertp = header which never worked if enabled in voipmonitor.conf file
- redesign threading which is now enabled by default and allows to use all available CPU cores. Memory is now fixed size and not causes crash. Communication between threads was optimized too. VoIPmonitor is now able to handle full 1Gbit traffic ~7000 simultaneous calls within one instance (tested on Xeon X5690 and Intel 1Gbit 82580 card using DNA driver). New options controlling threading - vmbuffer, rtpthreads, rtpthread-buffer
- automatically creates mysql tables in database 
- implement id_sensor option to diferentiate cdr between several instances saving to one database and one table
- implement getfile and gettshark file over tcp manager to support many sniffers - one database/GUI
- fix init.d script and pid file handling
- implement possibility to remove duplicate packets 
- no user action on database changes is needed 

version 5.1
===========

- redesign SIP REGISTER handling. register table is now showing only active REGISTRATIONS. Once it expires it is removed from the table to new sql register_state table. The register state table is used to store changes in registrations. New sql table register_failed is used to store all failed sip register - to not overload the table there is counter column which adds +1 for each failed register. The WEB GUI will follow and is not finished yet. 
- Allow all characters from ASCII table in telnunm capture rules which means that now it is possible to make prefix 00 or +420. 
- Implement SIP overlap dialing (used a lot in Germany, thanks Carsten Buchberger) 
- fix rare memory corruption / crashes presented in all versions - The callers display-name were not extracted, if it is not enclosed in Double-Quotes which is still valid according RFC
- Implement sniffing packets with two IP headers to support Acme SBC mirror feature
- Implement mysql port configuration option to connect to non standard 3306 port - in voipmonitor.conf and on command line
- fix natalias configuration issue - Allows to save only UDPTL (T.38) FAX packets instead of all RTP packets. It is for cases where only FAX is needed to save instead of RTP packets. PDF image creation from T.38 is available in upcoming WEB GUI 5.1 version.- to upgrade sql run cat cdrtable.sql.5.0-5.1 | mysql voipmonitor - this time it is instant because cdr table is not touched. 

version 5.0.6
=============

- fix memory corruption causing crashes 

version 5.0.5
=============

- fix IP and telnum based capture rules which were not working at all since version 5

version 5.0.4
=============

- Change in voipmonitor.conf localhost to 127.0.0.1 to fix mysql connection on centos for statically build packages
- increase call-id SIP header len from 32 to 128
- all SIP headers has to start right after new line. Otherwise the compact headers t: is maching Contact: 
- According RFC support To: header also if it is not in <> (<sip:....>) and thus correctly extract domain part 

version 5.0.3
=============

- do not load mysql filters if -c on command line. This fixes converting wav files which do not need mysql connection. from the version 5 connection is retrying undefinetly and without specifying -c voipmonitor will not continue to operate.

version 5.0.2
=============

- fix MySQL connection in case --config-file is not specified
- fix payload type in case the first RTP is DTMF or FLASH RTP event (payload 101) preventing GUI to play WAV

version 5.0
===========

- SQL schema changed. Upgrade script is in scripts/mysql_copy_4.2to5.0.php please refer to sniffer manual how to upgrade old 4.2 
- remove mysqlpp denendancy and use native mysql C library