Tuesday, September 28, 2021

How Skimmer Server Decides What to Spot - Beacons and More

How Skimmer Server Decides What to Spot – Beacons and More

Note: As changes/additions are made, I will add them in blue text.  Why blue?  Because it’s my favorite color.

I’ve just spent a couple of days in back-and forth e-mails with Alex, VE3NEA, and decided that it would be useful to clarify how both node-ops and station-ops wanting to spot or be spotted could maximize their chances. I apologize for the length of this, but there’s material here you’ll find nowhere else.  If you’re not an RBN node-op, you can skip right to number 3 below.

1.      First, for node-ops, there are a couple of basic considerations:

·        Make sure your CW Segments (or RTTY Segments, if you’re working with RTTY Skimsrv) are set to cover the frequencies you want, and exclude those you don’t.  That second part is particularly important for RTTY, which is very resource-intensive.  If you spend a bunch of decoders trying to copy FT4, FT8 or other non-Baudot transmissions with RTTY Skimsrv, you can easily swamp your PC.  Just look up the latest list of frequencies for those modes and create CW segments that exclude them; for example, you can write CW or RTTY Segments for 14000-14072 and 14083-14100 or 14083-14105 if you also want to capture the NCDXF beacons at 14100.  Do similar things on other bands, avoiding published FT4/8 bands and capturing the frequencies you want, either for QSOs or for beacons.  There’s no practical limit on the number of CW or RTTY Segments, but of course the frequencies they define must be within the coverage set by the band segments in use.

·        There is one other basic consideration if node-ops want to copy beacons that do not send the CQ keywords (CQ and TEST) as part of their transmissions.  You must tell CW Skimsrv, on its Telnet tab, to send all spots to the Aggregator, not just CQ spots.  There’s no reason not to do this, because the Aggregator will sort out beacons and make sure they are spotted.  More on this below.

2.      The second important consideration, again for node-ops, is whether the callsign of the station to be spotted is recognized by Skimsrv as likely valid, and therefore qualified to be passed on to the Aggregator. This recognition is based on three devices.

·        The watch list, file name watch.lst, is a text file stored in the Skimsrv or RTTYSkimsrv folders in C:\Users\[your username]\Appdata\Roaming\Afreet\UserData.  It is not editable from within CW or RTTY Skimsrv, but must be edited using a simple text editor like Notepad.  Note also that there is no default watch.lst - you have to create your own.  You might choose to put in it callsigns like those of the NCDXF beacons (4U1UN, etc.), to make sure they are spotted whenever they are decoded.

·        The validation level, which is set by the node-op in CW or RTTY Skimsrv

·        The pattern file, filename patt3ch.lst, which is stored in C:\Users\[your username]\Appdata\Roaming\Afreet\Reference.  It consists of one line for each known callsign pattern.  Here’s a snippet:

 

+ OH2@@

+ OH2@@@

  OH2@@@@

  OH3###@

  OH3@

+ OH3@@

+ OH3@@@

 

As you can see, each pattern is denoted by its first three characters, whatever they are, plus “wildcards” for additional numbers (#) or letters (@).

 

There’s one other important feature.  Each pattern can be shown either with or without a “+”.  The “+” means that the callsign pattern is among those more commonly seen on the air.  Listing of a pattern without the “+” means that it has been seen, but less often.  And finally, if a pattern does not appear, that means that it has not been seen at all.

An example: Say a station, call him BY1AA has been active on the air.  His callsign pattern, BY1@@, will be marked with a “+”.  If he has not been that active, it might not have the “+”, but it is still on the list. But now he gets a contest call with a single-letter suffix, like BY1A.  The pattern BY1@ is not on the list at all.  What that means is that his call will have to be decoded much more often before a spot is sent up to the RBN for distribution.

 

What does “more often” mean?  CW Skimsrv works by assigning a 50-hz wide decoder channel to each signal it hears.  It then “looks at” a constantly updating 256-character sample from that decoder, and if the callsign is repeated enough in that length of time, the station is spotted.

 

An initial patt3ch.lst file is distributed with CW and RTTY Skimsrv, and the organizers of the RBN have published updated versions at least every year, based on calls that have been heard in the time since the last list came out.  Sometimes, the organizers will release an updated pattern file when some special event occurs.  For example, when EA amateurs were permitted to have single-letter suffixes, the RBN distributed a revised pattern file with those callsign patterns.  RBN node-ops are notified whenever they start the Aggregator if the pattern file on the RBN server is different from the one they have currently.  There is an easy one-click option in the Aggregator to quickly and easily download the new pattern file.

 

·        The table below will explain why you need to know all this.  The interaction of the Watch list, the pattern file, and the validation level the node-op has chosen (ranging from Minimal to Paranoid) determines how many times Skimsrvs must decode a given callsign before “deciding” to accept it and send the spot on to the RBN.

 

                             Number of Repetitions Required to be Spotted

Pattern File/Validation

Minimal

Normal

Aggressive

Paranoid*

Watch list

1

1

2

2

+ in pattern file

2

2

4

-

No + but in file

2

3

4

-

Not in file

2

5

-

-

Not ITU prefix

-

-

-

-

 

3.       Now we get to the user part, which is pretty simple.  For users or beacon operators who want to make sure their transmissions result in spots, there are several cautions:

 

·        For non-beacons, be sure you send one of the keywords – TEST or CQ (FD, SS, NA and UP also work in appropriate settings) – at least once in each transmission.  Send your callsign at least twice.  When in doubt, send it 4 times.  It doesn’t necessarily have to be all in the same transmission, because of the length of the decoding buffer.

·        Make sure the CW is good – letter formation and spacing, and 3:1 dot/dash ratio.  Any errors here may manifest themselves as copying errors.  Use a memory keyer or software, if feasible, to be sure of this.  Skimsrv is pretty forgiving, but there are limits. Above all, avoid using half-spaces right before or right after your callsign. 

·        If you want to repeat a transmission right away and have the same station spot you again – in antenna comparisons, for example – you must QSY a half-KHz or so before making the next transmission.  Skimsrv will not spot you on or close to the same frequency more than once every 10 minutes.

·        When operating remotely, check to be sure your CW is clean as transmitted – Internet latency can garble your CW.

·        For beacons, and particularly ones that are in the normal CW parts of the bands, make sure that your call and frequency are on one of the five most commonly-used beacon lists:

https://www.keele.ac.uk/depts/por/50.htm

https://www.keele.ac.uk/depts/por/28.htm

http://www.newsvhf.com/beacons2.html

https://mmmonvhf.de/beacon/beacon.csv

http://www.qsl.net/wj5o/bcn.htm

 

The Aggregator downloads these files from the server every night and automatically updates its beacons.csv file.  It checks every spot it receives against the file and uploads those that it receives from Skimsrv or RTTY Skimsrv, as well as any spots with /B on the end of the callsign, regardless of the presence or absence of keywords.


Sunday, February 28, 2021

Info from N6TV on implementing CWSL on a Red Pitaya 122-16

 Everything you need to do is explained at the bottom of the CWSL_Tee web site:


But some of the links are broken, and you have to go down one level and click Download to actually get the links to the files.   Here are some direct links:



Important additional steps  In the SkimSrv2 folder:

Rename SkimSrv.exe to SkimSrv2.exe
Rename CWSL_Tee.dll to CWSL_Tee2.dll
Rename HermesIntf_xxxx.dll to HermestIntf_FFxx.dll 

Update CWSL_Tee.cfg:
   HermestIntf_FFxx
   64

To access either receiver with HDSDR, copy Extio_CWSL.dll to the HDSDR folder, and copy Extio_CWSL.dll to Extio_CWSL2.dll to access the second receiver.

Monday, November 16, 2020

Editing the start.sh file for 16-bit Red Pitaya on a Windows Computer

 Bjorn, SM7IUN has brought my attention to a potentially important caveat in editing the start.sh file to activate two entirely separate receivers on an RP-16.

The RP-16 is a Linux device, so the start.sh file must be compatible with Linux.  However, if you edit a Linux file using Windows Notepad, the edited file will come out with <CR><LF> at the end of each line, rather than the Linux standard <LF>.  This, in turn, could result in its no longer working under Linux.  While I have not yet had this problem with my RP-16s, it's worth knowing about.

Fortunately, there is an easy solution - Notepad++.  Written by Don Ho, it is freeware, and senses whether a file selected for editing follows the Linux convention or the Windows and ends lines appropriately.  A worthwhile precaution, I'd say. 

Sunday, November 15, 2020

How to Post Two Receivers' Output to the RBN under Two Different Receiver Names

As outlined in my post a couple of weeks ago, it is not possible, using a single Aggregator, to post spots from the two receivers in an RP-16 to the RBN under separate receiver names.  Fortunately, Dick Williams, W3OA, the author of the Aggregator, has figured out a work-around.  Here's his solution:

1)  Update Aggregator to version 6.2.  Aggregator will now show the name of its executable file in the title bar at the top of its window.  This enables you to determine which Aggregator instance the window refers to.

2) Aggregator was running from C:\Ham Stuff\Aggregator. I created a new folder, C:\Ham Stuff\Aggregator2, and copied Aggregator.exe from the original folder to the new one.  Also copy BADCALLS.txt if you are using that filter.

3) Rename Aggregator.exe in the new folder to Aggregator2.exe.

4) In your first Aggregator instance go to the “Secondary Skimmers” tab, find the line which connects to the second SkimSrv instance, click the “D” button, and uncheck the “Auto Connect?” box.

5) Again in your first Aggregator instance go to the “FT#” tab and uncheck the “Use?” box for each source which receives spots from the “Rig Numbers” on the right half of the FT#StartUp window, i.e SkimSrv2.  Then click the “Apply Changes” button.

6) In your second SkimSrv instance go to the “Operator” tab and update the entries there as necessary.  The callsign should be the SSID you want to use for that Skimmer.  (I used W3OA-2.)

7) Start your second Aggregator instance and go to the “Connections” tab to set the second SkimSrv instance as the “Primary Skimmer Connection” for this Aggregator instance.  While on this tab change the “Local User Port Number” so it's not the same as in the first Aggregator instance.

8) Complete setting up the second Aggregator instance paying particular attention to the “Spot Filters”, “Patt 3Ch.lst”, and “FT#” tabs.  On the “FT#” tab add the sources you disabled in the first Aggregator instance in step 5. Then click the “Apply Changes” button.

9) Return to the first Aggregator instance, go to the “Associate Pgms” tab, and add the second Aggregator instance.  

Ethernet/Internet requirements for 2-receiver Red Pitaya

 Ash KF5EYY/3V8 just reminded me of an additional requirement for running both receivers on an RP-16, to cover 8 bands each.  Each receiver requires about 75 Mbps of Ethernet bandwidth between the receiver and your PC, and it adds up.  This means that your PC must have a gigabit Ethernet port in order to receive two receivers worth of 8-band x 192 KHz data.

My PC has such a port, but like many it was already committed to my Internet router.  Fortunately, Netgear makes a very nice unmanaged Ethernet switch, the GS 308 which retails for under US$20 plus shipping.  All I had to do was plug my receiver, my PC, my KPA-1500 amp and my router into the switch, and it was up and running.  Whew!    

Monday, September 21, 2020

Part 2 - Using the Two Receivers in Your Red Pitaya


The RP-16 has a hidden, bonus capability – it can run a second, 8 bands x 192 KHz receiver alongside but independent of the first.  This capability has a lot of potential uses – I often use it for comparing two antennas.  I have also used it to run RTTY Skimmer Server along with CW Skimmer Server on a single antenna,  without having to resort to CWSL_Tee to share the output from a single receiver .  Many operators use it to expand coverage on CW or RTTY to as many as 11 or 12 bands at once, or to add FT4 or FT8 capability.

One caution, to be mentioned first for emphasis– if you run two Skimmer Servers on the two receivers of the RP-16, do not try to send spots from both to the Reverse Beacon Network.  You would think that by identifying the Skimsrv operator by two different callsigns, like W8QZR and W8QZR/3, for example, they would appear separately on the server.  The problem is that the RBN Aggregator is only capable of reporting one operator callsign to the RBN server, so each spot will be attributed to one call, and any stations that both receivers spot will appear twice.    This is a function of how the Aggregators communicate with the server and, at least for the moment, cannot be changed. 

Of course, this caution does not apply to running CW Skimmer Server on one receiver and RTTY Skimmer Server on the other, since their modes are different.  It also obviously does not apply to using one receiver on one set of bands, and the other on different ones, as you might do to optimize receiving or covering 6 meters along with HF. 

There are two options for the second receiver – it can either share the antenna with the first receiver, or it can use the second input on the RP-16 to connect a separate antenna.  

When you boot up the RP-16, it starts two separate receivers and connects them to two different IP addresses.  From there on, it’s up to you – you can connect a separate instance of Skimmer Server (see below), or any SDR that can use the Hermes_intf.dll.  The only one I have tested this with is HPSDR. 

To use a separate antenna on the second receiver, put the RP-16’s SD card in your reader and bring up the start.sh file for the hpsdr-compatible receiver in your text editor.  It is pretty short and simple, and will look like this: 

apps_dir=/media/mmcblk0p1/apps

source $apps_dir/stop.sh

cat $apps_dir/sdr_receiver_hpsdr/sdr_receiver_hpsdr.bit > /dev/xdevcfg

address=`awk -F : '$5="FF"' OFS=: /sys/class/net/eth0/address`

ip link add mvl0 link eth0 address $address type macvlan mode passthru

echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter

echo 2 > /proc/sys/net/ipv4/conf/all/rp_filter

$apps_dir/sdr_receiver_hpsdr/sdr-receiver-hpsdr eth0 1 1 1 1 1 1 1 1 &

$apps_dir/sdr_receiver_hpsdr/sdr-receiver-hpsdr mvl0 1 1 1 1 1 1 1 1 & 

All we are concerned with is the last line, and to activate the second receiver on its own antenna all we need to do is change each of the “1”s in that line to a “2”, So it reads "$apps_dir/sdr_receiver_hpsdr/sdr-receiver-hpsdr mvl0 2 2 2 2 2 2 2 2 &”.  Save the file to your SD card, and the second receiver is ready to go, on its own antenna. 

In order to run Skimmer Server on the second receiver, you’ll need to create a separate instance of Skimmer Server in your computer, with a different name, and also rename the executable.  What I did was to install it under the program folder name “Skimsrv 2”, and rename the executable from Skimsrv.exe to Skimsrv2.exe. 

There’s one final thing to be done before you start the two simultaneous receivers.  In the first Skimmer Server program folder, find the Hermes_Intf.DLL.  Rename it to explicitly recognize the first receiver.  The MAC address of that receiver is printed on top of its Ethernet connector.  You need to copy the right-most four characters of the MAC address, and then use them to rename the Hermes_Intf.dll file.  In my case, it became HermesIntf_7c93.dll.  Don’t forget the “_”. 

Now go into your second Skimmer Server directory and rename its HermesIntf.dll, in this case replacing the first two characters of the MAC address with “ff”.  Rename the HermesIntf.dll as you did above for the first.  In my case it became HermesIntf_ff93.dll. 

And now, finally, it’s ready to go.  Start CW Skimserv on the first receiver.  Go to its Skimmer tab and verify that it has the correct address, like this:

 

Now start the second instance of Skimmer Server and verify the same thing:

 


 

Now you can connect a second antenna to the second SMA input connector on the RP-16 (next to the first one), and you’re off and running. You can get data from either receiver using a Telnet program like Putty – be sure they have different Telnet addresses on the Telnet tab in Skimserv.  You can also use the Aggregator’s Skimmer Traffic tab to follow what each is spotting, but if you do that, be sure that you check the “don’t send spots to RBN server” box on the Connections tab (far left side) and set up the Secondary Skimmers tab with the right (different) Telnet port number to connect to the second instance of Skimsrv.

 

 

 

  





Wednesday, June 17, 2020

Using the Red Pitaya Stemlab 122.88-16 on the Reverse Beacon Network – Part 1


Using the Red Pitaya Stemlab 122.88-16 on the Reverse Beacon Network – Part 1

Introduction
The Red Pitaya STEMlab 122.88-16 (which I will call the RP-16 from here on out) reflects the company’s recognition of the needs of radio amateurs, and particularly those who are interested in contributing to the RBN.  Its 50-ohm input impedance, 16-bit dynamic range and other capabilities I’ll get into later, make it the ideal successor to the QS1R receiver that many of us used initially, but which is no longer available.

My purpose in writing this guide is to organize information, much of which is already scattered across the Internet, about various ways you can use the RP-16 to receive and/or spot CW, RTTY, and other digital modes (including FT8 and FT4).  In this respect, I must recognize the contributions of Bob Wilson, N6TV, who has worked tirelessly to help the RBN-OPS and Skimmertalk communities realize the potential of the RP-16.  Thanks also to Pavel Demin for his foundational programming of the receiver application for the RP-16; without him, it would be just a nice piece of multi-purpose lab equipment.

Getting the Red Pitaya-based receiver going
One caution going in – there has been a lot of discussion over the last few years about using various Red Pitaya versions on the RBN.  Be forewarned that most of that information, particularly as it refers to interaction with the file system on the RP, is obsolete unless it is specific to the RP-16.  For example, as Bob, N6TV discovered, “The home directory on Alpine is actually mounted on a temporary file system (like a RAM disk).  It goes away and is recreated from the archive on the SD Card every time you reboot the RP.  If you don't update the archive, any changes are lost.  This applies only to Alpine Linux, not Ubuntu.  Files in the /media/mmcblk0p1/ folder (including the "apps" folder, which is a symbolic link, like an alias) are actually on the SD card.”

Before you put the RP-16 into service with Skimmer Server (SkimServ), there are a couple of important considerations.  The first thing to know is that because it is intended as a multi-purpose instrument platform, the RP-16 has no filtering or protective circuitry between the antenna input and the Analog to Digital Converter (ADC).  This can cause all sorts of obscure interference problems.  For example, I had a couple of loud FM stations that raised the noise floor considerably on 15 meters.  There are a variety of HF low-pass filters on the market, typically with roll-off beginning at 70 MHz.  Another option is to use that old TVI filter that’s been in the basement forever.

The second caution has to do with Ethernet, how that the RP-16 communicates with your computer or network.  Because of the tremendous capabilities of the RP-16, it moves a LOT of data across the Ethernet connection to your computer.  For example, I’m currently running two 8-band receivers, and my PC says that the data flow to the two instances of SkimServ is 166 Mbps.  In other words, you’ll need a Gigabit Ethernet adapter on your PC.  The most trouble-free way I’ve found to manage this is to put an inexpensive Gigabit Ethernet switch in your station, and connect it to your RP-16, shack computer, and router.  The resulting performance is seamless and reliable.

For now, though, let’s just get it ready to run: The RP-16 requires both an operating system (Linux) and application(s), including the SDR itself.   It comes with a micro-SD card and adapter to fit your PC’s SD card slot (or one of the many SD card readers on the market).


The first thing you need to do is format the card in FAT32, which is not what typically comes on the card.  It is not the same as FAT.  Then find Pavel Demin on Github (specifically, go to https://pavel-demin.github.io/stemlab-sdr-notes/sdr-receiver-hpsdr/ There’s a huge amount of information there on how Pavel turned the RP-16 into a software defined receiver (SDR) compatible with the Hermes standard.  You will also find links there to a zipfile containing the bootable SD card image, which you will want to unzip and copy onto the SD card.  It includes both the Alpine Linux operating system and several receiver and transceiver programs for various modes.

Before you can run either CW or RTTY Skimmer Servers on the RP-16, you also need to download HermesIntf-18.5.22.zip or later from K3IT’s Github page <https://github.com/k3it/HermesIntf/releases>.  Extract the Zip file and save HermesIntf.dll in the Skimmer Server program folder (usually C:\Program Files (x86)\Afreet\SkimServ\).  Rename the DLL to HermestIntf_xxxx.dll where xxxx is the last 4 characters of the RP’s MAC address, which is printed on the label on top of the Ethernet connector. 

As it comes from Demin, the SD card has a number of different applications on it.  You want only one, apps/sdr_receiver_hpsdr.  To start the receiver automatically at boot time, copy its start.sh file from the apps/sdr_receiver_hpsdr folder to the topmost directory on the SD card.  This can be done either on Windows, before you insert the card into the RP, or by entering the following three Linux commands in sequence, after logging in to the RP command prompt with PuTTY (default id:  root,  default password: changeme):

Insert the SD card in RP-16 SDR and connect the power. If the operating system and the HPSDR-compatible receiver app are properly installed on the SD card, when you connect power to the right-most micro-USB port, the RP-16 will start booting up, which typically takes 30 seconds or so.  During boot-up, the blue and green LEDS next to the RP-16’s Ethernet connector will be on steadily, and a yellow LED at the opposite end of the array will flash roughly once a second.  When it stops, you’re ready to move on.  The blue and green LEDs stay lit so long as power is applied.

In the next installment I’ll tell you how to activate and use the second complete receiver in the RP-16.  Stay tuned.