Testing Spot Quality Filters

It's been a busy and rewarding couple of weeks.  Sometimes, I'm quite overwhelmed by the willingness of hams to invest large amounts of time and intelligence in advancing our hobby, not for any personal gain but simply because they care and enjoy what they are doing. 

This is one such case.  The AR Cluster V6 Telnet server at the RBN has just been replaced with a beta version that provides quality scores for each RBN spot, as well as filters enabling users to apply those scores to limiting the spots they receive.  Note that if you don't set any of the new filters, the node will continue to function as it always has, except for the addition of a validity code as part of the comment on each RBN spot.  All existing filters will continue to work as before.  The full story is at <http://www.ab5k.net/ArcDocsVer6/UserManual/ArcDx_CT1BOH.htm>.

Special thanks to CT1BOH, who did extensive analysis of RBN data and developed the innovative algorithms being used, and to AB5K, for the hard work of coding and testing the filters for incorporation in this new version of his cluster software.

During the beta period, users will see a set of new tags in the comment field of spots coming from the RBN ARC6 Telnet server.  These will not be a feature of the finished version, so software that relies on a particular format in the Comment field of spots will not have to be modified.  Users will see the quality tag, but in order to filter using them, the node they are connected to must be running the beta server software as well.  Users can test with the RBN node at arcluster.reversebeacon.net, port 7000.  Node sysops who want to try the beta are encouraged to contact AB5K.

These are the quality tags:

? - Not yet verified.  The first and/or second identical spot of the same station, including erroneous spots of callers 
V - Valid, meaning that at least three identical spots (call and frequency) have been posted by Skimmers worldwide
Q - QSY?, meaning that it is the first and/or second spot of a station on a new frequency, where spots of that station were previously verified. Sometimes, this will be a legitimate QSY, but the tag may also indicate an I/Q image of a good spot.
B - Bust, based on whether a new spot is enough like ones already seen and tagged Valid, except for a difference in the callsign.  This is based on a really clever applied math concept called the Levenshtein distance.  Google for more info.
. - Unique, meaning that there are only one or two stations currently spotting stations in a given country.  Will often change to V if more stations spot it, but in the meantime you won't miss that P5 because only one RBN station heard it.

You can filter so that you get only Valid spots, or so that you can block all busted spots, or so that you get no Q spots until they are verified (so I/Q images will not come through).  You can even tell the cluster node to let through spots with a "." tag, so you don't miss the really rare one who is only spotted by one or two stations.  Full info and examples at the URL above.

I do not assume that this is the last word on improving RBN spot quality.  Beta means beta.  Please experiment, see what you think, and let me know.  One particularly fruitful line of inquiry would be to compare the arrival time of unfiltered spots and spots that have been judged Valid, to note instances when an apparently legitimate spot doesn't get through or is judged to be a bust, or when a bust is not caught. 

In this connection, in testing we have noticed that sometimes a Busted call will be mistakenly judged Valid, because there are more busts than good spots in a time window.  Often, this seems to be due to spacing errors (RN4ZR for N4ZR), frequent omission of portable designators by ops (N4ZR instead of P5/N4ZR), and PTT cutting off the first dit (E4ZR for N4ZR)  

Please send your comments/reports to me, and I'll see to their onward distribution.  Do not send them to this reflector!


73, Pete N4ZR