Let's start with wget
![Smile :-)](./images/smilies/icon_smile.gif)
Moderators: marcelr, TheHogNL, Toonz
Should be easy; the site is an almost exact copy of the one of cure afvalbeheer; will post the download scripts and cron install manual later today.QuasaR wrote:Can you implemente support for Cyclus ?
https://afvalkalender.cyclusnv.nl/form (2841 SP / 44)
Code: Select all
###############################################################################
How to download "afvalwijzer" data from a https website, using wget and cron
20170513, marcelr, first draft
###############################################################################
Recently, Toonz created an app which displays the waste collection agenda for
a specific address on toon.
From version 1.3 onwards, it also supports cure-afvalbeheer (active in the
Eindhoven area).
The waste collection agenda provided by this company is available as an .ics
file, from their website. The website is served as a https service, offered
through service port 443 (secure http or https).
## getting acccess to files served through https (port 443) ##
Toon has no support for this protocol, by default. So, to gain access to this
site, the following is required:
1: (Access to) a rooted toon, with working ssh and (preferably) sftp servers.
2: Install package for wget, with ssl support. Available from the Downloads
thread on this forum.
3: Some proficiency in using vi.
What to do:
1: Upload wget with ssl support (wget_1.12-r8.2_qb2.ipk) to toon and install:
opkg install wget_1.12-r8.2_qb2.ipk
2: Open port 443 in the firewall:
Locate the file /etc/default/iptables.conf and add the following line:
-A HCB-INPUT -p tcp -m tcp --dport 443 --tcp-flags SYN,RST,ACK SYN -j ACCEPT
3: Restart iptables by issueing the command:
/etc/init.d/iptables restart
## Find the right URL for your waste agenda and adapt the download script ##
1: The agenda for the waste collection is given in the cure website:
Fill in your address details (post code, house number) and click
NAAR MAAND- EN JAARKALENDER > -> ZET INZAMELDATA IN UW AGENDA
Locate the link to your agenda and copy it into the "get_data_from_cure.sh"
script (it has the following shape):
https://afvalkalender.cure-afvalbeheer.nl/ical/0yyyy000000xxxxx
#! /bin/sh
#
# get_data_from_cure.sh
#
# script to download calendar data from waste management company Cure
# (Eindhoven area, NL)
#
# call: get_data_from_cure.sh <address code>
#
# The address code is found at the cure website.
# fill in your address details (post code, house number) and click
# NAAR MAAND- EN JAARKALENDER > -> ZET INZAMELDATA IN UW AGENDA
#
# At this page, copy the link
# https://afvalkalender.cure-afvalbeheer.nl/ical/<whatever-your-number-is
# to the command line after the call to this script:
# get_data_from_cure.sh <address code>
#
# Hit <enter> and your personal calender is downloaded to the directory
# defined in $DIR_OUT.
# The address code is linked to your address, not to a time, and thus remains
# the same, so you can repeat downloads from the same URL, and get updates for
# your calendar.
#
# another option is to paste the URL into this script, replacing the first
# argument ($1) of this script, in the definition of $URL below.
WGET=/usr/bin/wget
FILE_OUT=waste_calendar.ics
DIR_OUT=~/waste/
#URL=$1
URL=<paste URL here>
mkdir -p $DIR_OUT
$WGET --no-check-certificate -O /$DIR_OUT/$FILE_OUT $URL &> /dev/null
# end of script
2: On toon, create the directory /root/bin:
mkdir -p /root/bin
3: Upload the "get_data_from_cure.sh" script to toon, put it in /root/bin, and
make sure it is executable:
chmod 755 /root/bin/get_data_from_cure.sh
4: Test by issueing the command /root/bin/get_data_from_cure.sh
This should give you a file called waste_calendar.ics, in the directory
/root/waste
## Automation of the calendar download ##
To automate regular execution of programs, unix systems (like toon) use a
utility called cron. Cron is a daemon process which runs in the background,
and checks every minute whether something from the crontab file needs to be
done. The crontab file is a file which sets the rules for cron, what to
execute, and when. It is edited with the program crontab.
To automate waste agenda download, do the following:
1: Download and install cron_3.0pl1-r8_qb2.ipk
2: Edit the crontab for user root, by issueing the following command:
crontab -e
This opens an editor (typically vi), paste the following into the file:
# crontab for root
# The time and date fields are:
#
# field allowed values
# ----- --------------
# minute 0-59
# hour 0-23
# day of month 1-31
# month 1-12 (or names, see below)
# day of week 0-7 (0 or 7 is Sunday, or use names)
#
SHELL=/bin/sh
# download .ics file for "afvalwijzer" app, every third day of the month,
# at 4:15 in the morning:
15 4 3 * * /root/bin/get_data_from_cure.sh
and save. You may omit the lines starting with a hash mark (#).
3: Start the cron daemon, by issueing:
/etc/init.d/cron start
## Getting the information on toon ##
1: Install the app wastecollection v1.3 or later.
2: Edit the file WastecollectionApp.qml, to have the property wasteCollector
reflect your waste collector:
property string wasteCollector : "cure-afvalbeheer.nl"
and save.
3: Add the wastecollection app to the file /HCBv2/qml/qb/base/Globals.qml.
4: Restart qt with killall qt-gui
5: All done, you should now be able to install a tile with your waste
collection data from toon's tile selection menu.
Code: Select all
#! /bin/sh
#
# get_data_from_cure.sh
#
# script to download calendar data from waste management company Cure
# (Eindhoven area, NL)
#
# call: get_data_from_cure.sh <address code>
#
# The address code is found at the cure website.
# fill in your address details (post code, house number) and click
# NAAR MAAND- EN JAARKALENDER > -> ZET INZAMELDATA IN UW AGENDA
#
# At this page, copy the link
# https://afvalkalender.cure-afvalbeheer.nl/ical/<whatever-your-number-is
# to the command line after the call to this script:
# get_data_from_cure.sh <address code>
#
# Hit <enter> and your personal calender is downloaded to the directory
# defined in $DIR_OUT.
# The address code is linked to your address, not to a time, and thus remains
# the same, so you can repeat downloads from the same URL, and get updates for
# your calendar.
#
# another option is to paste the URL into this script, replacing the first
# argument ($1) of this script, in the definition of $URL below.
WGET=/usr/bin/wget
FILE_OUT=waste_calendar.ics
DIR_OUT=~/waste/
#URL=$1
URL=<paste URL here>
mkdir -p $DIR_OUT
$WGET --no-check-certificate -O /$DIR_OUT/$FILE_OUT $URL &> /dev/null
# end of script
Code: Select all
# crontab for root
# The time and date fields are:
#
# field allowed values
# ----- --------------
# minute 0-59
# hour 0-23
# day of month 1-31
# month 1-12 (or names, see below)
# day of week 0-7 (0 or 7 is Sunday, or use names)
#
SHELL=/bin/sh
# download .ics file for "afvalwijzer" app, every third day of the month,
# at 4:15 in the morning:
15 4 3 * * /root/bin/get_data_from_cure.sh
Hi QuasaR,marcelr wrote:SHELL=/bin/sh
# download .ics file for "afvalwijzer" app, every third day of the month,
# at 4:15 in the morning:
15 4 3 * * /root/bin/get_data_from_cure.sh
Tnx Toonz.Toonz wrote:Hi QuasaR,marcelr wrote:SHELL=/bin/sh
# download .ics file for "afvalwijzer" app, every third day of the month,
# at 4:15 in the morning:
15 4 3 * * /root/bin/get_data_from_cure.sh
I noticed your waste calender contains only data for the next 4 weeks (CureAfvalbeheer gives more entries in the future).
You might consider retrieving the file every 2 weeks instead of once a month.
Regardz,
Toonz