Archiv des Autors: hamster

awk script to remove objects from a pdf

First uncompress the pdf if it is compressed with pdftk:
pdftk myfile.pdf output unc.pdf uncompress

Then remove all objects that contain the keywords PDF-XChange|pdfxviewer.com|PDFXCViewer20|Click to buy NOW:

awk '
    BEGIN {
        found=0
    }
    {
    if ( $0 ~ /^[0-9 ]+obj/ ) {
        objectFound=1;
        objectLineCounter=0;
        objektZeilen[objectLineCounter]=$0;
        objectLineCounter++;
    } else if (objectFound == 1) {
        objektZeilen[objectLineCounter]=$0;
        if ( $0 ~ /PDF-XChange|pdfxviewer.com|PDFXCViewer20|Click to buy NOW/ ) {
            found=1;
        }
        if ( ( $0 ~ /endobj/ ) && ( found == 0 ) ) {
            for (i=0; i<length(objektZeilen); i++) {
                print objektZeilen[i];
            }
            delete objektZeilen;
            objectFound=0;
            found=0;
        }
        if ( ( $0 ~ /endobj/ ) && ( found == 1 ) ) {
            delete objektZeilen;
            objectFound=0;
            found=0;
        }
        objectLineCounter++;
    } else {
        print $0
    }
    }
' unc.pdf > test.pdf

Recompress and repair pdf with pdftk:
pdftk test.pdf output comp.pdf compress

Too bad it also removed the OCR layer. I couldn’t find out which layer is responsible for the OCR.

extract all rars in all subdirectories with find

Findet alle multipart rars in allen subdirectories von . außer dir1 und dir2 und entpackt in dem Ordner die Dateien.


find . \( -path "./dir1" -o -path "./dir2" \) -prune -o -name '*.r00' -execdir unrar x {} \;

Wenn schon Dateien vorhanden sind, die nicht überschrieben werden sollen:


find . \( -path "./_GameRetroPacks" -o -path "./_Konsol" \) -prune -o -name '*.r00' -execdir bash -c "echo E | unrar x {}" \;

Löschen aller Rardateien in allen Unterordnern:


find . -regextype posix-extended -iregex ".+\.r[0-9ar]{2}" -exec rm {} \;

Create a guest WLAN with OpenWrt

It’s enough to do it like this tutorial says:
https://wiki.openwrt.org/doc/recipes/guest-wlan-webinterface

If you want a log of the surfing history for maybe avoid being held responsibly for the things your guest does (Störerhaftung) you can log the DNS queries:
https://superuser.com/questions/632898/how-to-log-all-dns-requests-made-through-openwrt-router/897413#897413

Now you need a rsyslog server that collects those logs:
https://kimkil.nietsniehu.net/raspberry-pi-as-a-log-server-with-rsyslog-2/

Ubuntu and a custom build zfs and spl from git

I wanted to try the 0.7.0-rc3 from git. So I cloned from git and changed to the zfs and spl version 0.7.0-rc3 tag.

Since ubuntu ships with a prebuilt zfs and spl my custom packages didn’t work.
It always loaded the prebuilt zfs and spl from the kernel modules directory but not the one I built.
I figured out a way to make it work nonetheless.

– check your currently running zfs module:

# modinfo zfs
filename:       /lib/modules/4.8.0-41-generic/kernel/zfs/zfs/zfs.ko
version:        0.6.5.8-0ubuntu4.1

– remove kernel mods (make sure to have all zfs pools unmounted, otherwise it will fail):

rmmod zfs zunicode zcommon znvpair spl zavl

– remove kernel module from your current running kernel:

rm -r /lib/modules/$(uname -r)/kernel/zfs

– build deb packages for your currently running ubuntu (see their wiki):

cd $where_your_spl_from_git_resides
make clean; ./autogen.sh
cd $where_your_zfs_from_git_resides
make clean; ./autogen.sh
cd $where_your_spl_from_git_resides
./configure; make deb
for file in *.deb; do gdebi -q --non-interactive $file; done
cd $where_your_zfs_from_git_resides
./configure; make deb
for file in *.deb; do gdebi -q --non-interactive $file; done

– do a modprobe:

modprobe zfs

– check again the running zfs module:

# modinfo zfs | head -2
filename:       /lib/modules/4.8.0-41-generic/extra/zfs/zfs/zfs.ko
version:        0.7.0-rc3

After that it worked just well. I guess it has to be done for every new kernel release. If there is another better way instead of deleting modules from the kernel modules directory to make it work, please share.

PulseAudio erlaubt es nicht das Audio Ausgabegerät zu wechseln

Vielleicht liegt es am pavucontrol, dass nicht umgeschaltet werden kann. Nach ein wenig googeln findet man Kommandos, wie man die Audioausgabe über Konsole abändern kann.
pacmd move-sink-input 2 1 ging nicht. Fehler ist „Moved failed.“
Weiterer Details verrät:

hamster@hamsterkaefig:~$ pacmd list-sink-inputs
1 sink input(s) available.
    index: 2
  driver: <protocol-native.c>
  flags: DONT_MOVE START_CORKED FIX_RATE 
  state: RUNNING
  sink: 2 <alsa_output.pci-0000_00_14.2.analog-stereo>
  volume: front-left: 65536 / 100% / 0,00 dB,   front-right: 65536 / 100% / 0,00 dB
          balance 0,00
  muted: no
  current latency: 50,91 ms
  requested latency: 23,22 ms
  sample spec: float32le 2ch 44100Hz
  channel map: front-left,front-right
               Stereo
  resample method: copy
  module: 14
  client: 14 <Tropico5>
  properties:
    media.role = hex:
    phonon.streamid = hex:
    media.name = "Playback Stream"
    application.name = "Tropico5"
    native-protocol.peer = "UNIX socket client"
    native-protocol.version = "31"
    application.process.id = "30530"
    application.process.user = "hamster"
    application.process.host = "hamsterkaefig"
    application.process.binary = "Tropico5"
    window.x11.display = ":0.0"
    application.language = "de_DE.UTF-8"
    application.process.machine_id = "96177112fcaef6926d04e5b80000002e"
    application.process.session_id = "c2"
    module-stream-restore.id = "sink-input-by-application-name:Tropico5"

Dort steht bei flags: "DONT_MOVE"
Danach gegoogelt findet man die Lösung z.B. hier und hier

Man erstellt eine Datei im eigenen Home namens .alsoftrc mit Inhalt:

[pulse]
allow-moves=yes

Raspberry Pi reset wireless lan to solve problems with no connection

I found a script on http://askubuntu.com/a/593589 which tried to reset the wlan interface in various ways. I modified that script a little with and installed in on cron and let it execute each minute (*/1 * * * * /root/bin/resetWifi.sh)

resetWifi.sh:


#!/bin/bash
# program to check wifi and reset if not running
if [[ ! -z "$(ps waux | grep $0)" ]]; then exit 0; fi
exec 2>&1 1> >(tee -a $HOME/wificheck.log)
GATEWAY=192.168.1.1
IWCONFIG_BIN=$(which iwconfig)
RFKILL_BIN=$(which rfkill)
PING_BIN=$(which ping)
DEVICE=$(iwconfig 2>/dev/null | grep 802 | awk '{print $1}')
function isPingWorking {
        if ${PING_BIN} -c 1 ${GATEWAY} >/dev/null 2>&1 ; then
                exit 0
        else
                echo "didn't work :("
        fi
}
while true; do
        isPingWorking
        # Failed, try to reset wifi - sometimes works ok
        date
        ifdown ${DEVICE}
        sleep 1
        ifup ${DEVICE}
        sleep 10
        isPingWorking
        echo "turn wlan stick power off... "
        ${IWCONFIG_BIN} ${DEVICE} txpower off
        sleep 3
        ${IWCONFIG_BIN} ${DEVICE} txpower auto
        isPingWorking
        echo "use rfkill to reenable wlan stick... "
        ${RFKILL_BIN} list
        ${RFKILL_BIN} block wifi
        sleep 3
        ${RFKILL_BIN} unblock wifi
        isPingWorking
done
exit 0

Test if you have rfkill installed. The other bins should be available on most default installations.

download from vitalsource

login and save session cookie:

wget --keep-session-cookies --user-agent=Mozilla/5.0 --save-cookies cookies.txt --post-data 'user%5Bemail%5D=someone%40someplace.com&amp;user%5Bpassword%5D=asdf&amp;return=https%3A%2F%2Fevantage.gilmoreglobal.com%2F%23%2F&amp;failure=https%3A%2F%2Fevantage.gilmoreglobal.com%2F%23%2Fuser%2Fsignin%2Ffailure%2Fsomeone%2540someplace.com&amp;jigsaw_brand=evantage' https://jigsaw.vitalsource.com/login

„print“ the whole page and look for the page with „print“ in it:
https://jigsaw.vitalsource.com/api/v0/books/somebook/print?from=chapter-1&amp;to=chapter-end

This will give you a html with all picture links in it. I saved it as links.txt
LINKS=$(sed -re '/src/!d' -e '/.js/d' -e '/.css/d' -e 's#.*src="(.*?)".*#https://jigsaw.vitalsource.com/\1#' -e 's#800#2048#' -e 's#%20# #' links.txt)
counter=1
for i in $(echo $LINKS); do
wget --load-cookies cookies.txt ${i} -O "$(printf "%03d" ${counter}).jpg"
((counter+=1))
done

If printing isn’t available, it is possible to iterate through the pages and get the image source out of the html. Open the first page and look at the frame source code for something like „https://jigsaw.vitalsource.com/books/HK758SH-00-DATASH-E/pages/348102883/content“. The number 348102883 can be iterated.
You get the image source link with:


wget --header="Accept-Encoding: compress, gzip" --header="Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" --user-agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/53.0.2785.143 Chrome/53.0.2785.143 Safari/537.36' --ignore-length --keep-session-cookies --save-cookies cookies.txt --load-cookies cookies.txt 'https://jigsaw.vitalsource.com/books/HK758SH-00-DATASH-E/pages/348102883/content' -O - | gunzip | sed -re '/src/!d' -e '/.js/d' -e 's#.*src="(.*?)".*".*?".*#https://jigsaw.vitalsource.com/\1#' -e 's#800#2000#' -e 's#%20# #'

If there are some numbers missing there is a 302 status code. Run in a loop and first check with

wget --keep-session-cookies --save-cookies cookies.txt --load-cookies cookies.txt -S "https://jigsaw.vitalsource.com/books/HK758SH-00-DATASH-E/pages/348102884/content?create=true" 2&gt;&amp;1 | grep "HTTP/" | awk '{print $2}' | head -n1

if it is 200 or 302. If 200 you can get the correct image source link.

Film aus einzelne Filmsegmente runterladen und zusammenführen von Arte

  • .m3u8 Datei finden und herunterladen
  • überflüssige Informationen entfernen
    sed -i -e '/^#/d' -e '/^$/d' index_1_av.m3u8
  • parallel die Dateien herunterladen
    cat index_1_av.m3u8 | parallel --gnu "wget -c {}"
  • Dateien umbenennen, damit in der richtigen Reihenfolge zusammengefügt wird
    for i in *.ts; do extracted_number=$(sed -re 's/segment([0-9]*)_.*/\1/' <<<"$i"); mv "$i" "segment_$(printf "%04d" ${extracted_number}).ts"; done
  • zu einem Film zusammenfügen
    cat segment_* > film.ts

Adblock for Jolla (SailfishOS)

Since there is no adblock for Jolla in their shop and I was pissed in some apps about too annoying ads I thought about doing an adblock app for Jolla. Long story short, I couldn’t figure out how to do an adblock app but how I could do a systemwide adblock solution via a proxy (privoxy). The bad thing is that you have to manually install that rpm and it just can be stopped via root shell (systemctl stop privoxy). On the other hand I tried to make it as independent as possible for need of user action. It searches for new adlbock lists if your Jolla is connected to WLAN and otherwise it just hangs in there and waits for it. There is no more to do than installing and sometimes being in a WLAN to get adblock lists updated.

If you want to try, go ahead: privoxy-3.0.23-1.noarch
sha256: 23ed9c50e9c88531318ed12bee68da033def73f41749e55eeceba9853c22b85c

Here the scripts for building: https://github.com/hamsterbacke/SailBlock