Backing Linux up to Blu-Ray

Reference link (link)

  • truncate –size=25GB /big/disk/mybr.udf
  • mkudffs /bigdisk/mybr.udf
  • mkudffs /bigdisk/mybr.udf
  • mount -oloop,rw /bigdisk/mybr.udf /mnt/test
  • growisofs -speed=1 -Z /dev/dvd=/bigdisk/mybr.udf (grow the iso to 25gb)
  • Install deps:  yum -y install cdw udftools

Some links to help learn how to do this:

  • http://allgood38.io/burn-bluray-data-disks-on-linux-minimize-coasters.html
  • http://www.troubleshooters.com/linux/blu-ray-backup.htm
  • http://linuxhacksandfixes.blogspot.com/2013/02/recording-data-blu-ray-from-command-line.html
  • block added

Crashplan setup

Diagnosing

The steps below can confirm whether or not memory usage is causing the symptoms described above.

  1. Navigate to the log folder for your operating system:
    • Windows XP: C:\Documents and Settings\All Users\Application Data\CrashPlan\log
    • Windows Vista, 7, 8: C:\ProgramData\CrashPlan\Log
    • Mac OS X: /Library/Logs/CrashPlan
    • Linux (default): /usr/local/crashplan/log
    • Solaris (default): /opt/sfw/crashplan/log
  2. Open each service.log.# file in a text editor and look for the phrase “OutOfMemoryError”:
[09.23.12 22:33:02.273 ERROR   QPub-BackupMgr       backup42.service.backup.BackupController] OutOfMemoryError occurred...RESTARTING! message=OutOfMemoryError in BackupQueue!
  1. If you see this message, adjust the maximum amount of memory that the CrashPlan app can use by following the steps in the recommended solution for your operating system.

Considerations

 

  • CrashPlan uses memory up to the maximum allotted. The maximum is used only if it is needed.
  • Code42 typically recommends allocating 1 GB (1024 MB) of memory per 1 TB of storage (or per 1 million files).​ Although CrashPlan only requires approximately 600 MB of memory per 1 TB of storage (or per 1 million files), our recommendation is intended to account for growth in your file selection.

Sample Recommendations    

Backup Selection Size Recommended Memory Allocation (MB)
Up to 1 TB or up to 1 million files 1024 (default)
1.5 TB or 1.5 million files 1536
2 TB or 2 million files 2048
2.5 TB or 2.5 million files 2560
3 TB or 3 million files 3072

Memory Limitations

You should not allocate more memory to the CrashPlan app than the maximum supported by your computer. For example, if your computer has 8 GB of RAM, you shouldn’t allocate more than 8 GB (8192 MB) to the CrashPlan app.

Windows 32-bit Operating Systems
On computers running a 32-bit Windows operating system, programs are limited to a maximum of 2 GB of RAM. After accounting for other resources required by your system, this limits CrashPlan’s maximum memory allocation to approximately 1200-1500 MB. If you are running a 32-bit operating system, we recommend allocating a maximum of 1536 MB of memory to CrashPlan. Alternatively, you can remove files from your backup selection to reduce the size of your backup archive.

 

Recommended Solution

Windows

  1. Stop the backup engine
  2. Open the Start menu
  3. Locate the Notepad program, right-click it, and select Launch as Administrator.
  4. Use File > Open to navigate to C:\Program Files\CrashPlan\
    • If you don’t see CrashPlanService.ini, change the filter in the bottom right from Text Document (*.txt) toAll Files.
  5. Open the file CrashPlanService.ini
  6. Find this line in the file:
    -Xmx1024m
  7. Change the number to a larger value, depending on the size of your file selection.
    • See guidelines above for recommendations.
    • For example, if you are backing up 1.5 TB of data, we recommend setting this line to -Xmx1536m
  8. Start the backup engine

Mac OS X

 

You will use Terminal for the steps below. To open the Terminal Application:

  • Using Finder, go to: Applications > Utilities > Terminal
  • Using Spotlight, search for Terminal
  1. Stop the CrashPlan service by entering this command:
    sudo launchctl unload /Library/LaunchDaemons/com.crashplan.engine.plist
  2. When prompted, enter your password, then Enter.
    Note: The password doesn’t display as you enter it.
  3. Run the following command to open a command line text editor:
    sudo nano /Library/LaunchDaemons/com.crashplan.engine.plist
  4. Using the arrow keys on your keyboard, find the line that contains the following:
    -Xmx1024m
  5. Change the number to a larger value, depending on the size of your file selection.
    • See guidelines above for recommendations.
    • For example, if you are backing up 1.5 TB of data, we recommend setting this line to -Xmx1536m
  6. Press Control+X to exit.
  7. Press Y to confirm the changes.
  8. You’ll see the prompt “File Name to Write”. Press Return to save to the existing location.

 

  1. Start the backup engine by typing:
    sudo launchctl load /Library/LaunchDaemons/com.crashplan.engine.plist

Linux / Solaris

  1. Stop the backup engine
  2. Using your favorite Terminal based editor, open for editing the file/usr/local/crashplan/bin/run.conf
  3. Locate the line that begins SRV_JAVA_OPTS
  4. Locate this argument:
    -Xmx1024m
  5. Change the number to a larger value, depending on the size of your file selection.
    • See guidelines above for recommendations.
    • For example, if you are backing up 1.5 TB of data, we recommend setting this line to -Xmx1536m
  6. Start the backup engine

Alternative Solutions

CrashPlan app version 3.6.3 and later

Solution 1: Increasing Memory Limit With Manual Commands

 

You can use the CrashPlan app’s manual commands to allocate more memory. You must be able to open the CrashPlan app to use this solution.

Steps

  1. Open the CrashPlan app
  2. Double-click the CrashPlan logo in the upper-right corner
  3. Enter the following command, using a value appropriate for the size of your backup selection (for example, 1536 for a 1.5 TB selection):
    java mx 1536, deauthorize, restart
    The CrashPlan app will close
    Deauthorize app
  4. Open the CrashPlan app​ and sign in
    CrashPlan resumes backing up with the increased memory allocation

 

Solution 2: Increasing Memory Limit After Reinstalling Your CrashPlan App

You can reinstall the CrashPlan app and use manual commands to fix the problem. This solution can work even if you can’t open the CrashPlan app.

Uninstalling the CrashPlan app removes most log information. Our Customer Champions use logs to help troubleshoot problems, so you may want to contact our Customer Champions before uninstalling. Uninstalling the CrashPlan app does not remove your CrashPlan archive, so your information is safe while you’re reinstalling.

Steps

  1. Uninstall the CrashPlan app
  2. Reinstall the CrashPlan app
  3. Open the CrashPlan app, but do not sign in yet
  4. Double-click the CrashPlan logo in the upper-right corner
  5. Enter the following command, using a value appropriate for the size of your backup selection (for example, 1536 for a 1.5 TB selection):​
    java mx 1536, restart
    The CrashPlan app will close
    Finding the Command window in the CrashPlan app
  6. Open the CrashPlan app
  7. Sign in to the CrashPlan app and back up normally

Amanda Home Backup Server *WIP*

Based off this link.

I want to backup all my linux and windows system to a central place, using a mix of BluRay / Tape for long term storage, and external USB disks for near term bulk storage.  The tapes and bluray would basically for if the usb drive decided to die.

Backing up a windows client to amanda:

http://wiki.zmanda.com/index.php/Zmanda_Windows_Client

 

References:

http://www.harkness.co.uk/services/backup.html

Home Server Amanda Backup Server

http://wiki.zmanda.com/index.php/Tapetype_definitions

http://www.amanda.org/

https://wiki.zmanda.com/index.php/Getting_Started_with_Amanda

https://www.howtoforge.com/centralized-backup-server-with-amanda-on-centos

https://www.howtoforge.com/disk-backup-with-amanda-on-debian-lenny

http://www.amanda.org/download.php

 

MPD – A Brief Guide

The Music Player Daemon, aka MPD, is a great little application to handle music on your linux computer. Until recently (when Linux really started to take off and gain popularity), MPD has been notoriously difficult to get to work properly. I decided to write this small intro to MPD after having used it for about 6 months. This is a little guide to it to get you started on using MPD as your music manager.

The World of MPD

First, what MPD is:

  • MPD keeps track of your music library. It does this by defining a music directory, to which you can easily add symlinks to all the various places where you store your music on your computer.
  • MPD tracks changes to your music library (any time an mp3 is removed, or a new album added, you simply update the MPD database file — which takes three or four seconds at most).
  • MPD can store playlists.
  • MPD can play songs, and output this information to various devices, such as your physical speakers (usually via your alsa sound drivers, but it can be any audio driver on your system), or even a http port for remote web listening (via icecast).

What MPD is NOT (what most people get confused about):

  • MPD cannot by itself determine: which songs are played and in what order. Nor does it come with any controls such as PAUSE, STOP, NEXT TRACK, etc. MPD, quite simply, can either: play a song, or do nothing while it is running. MPD’s behavior is controlled by clients, such as GMPC, Sonata, etc (my personal favorite is ncmpcpp).

The downside of this dichotomy between what MPD is and what it is not, is that you must install two applications to get the same functionality as any standard multimedia player like iTunes. However, there are many, many advantages of splitting up music-playing from music-controls. First, mpd will hardly crash and will be error-free due to its simple nature. Second, you gain the ability to access your large music collection from other computers in your home network without much fuss. This means that you can do things like: make your laptop act as a remote control for your desktop’s music output (the desktop’s speakers), or even create multiple MPD instances for specific purposes simultaneously (e.g., alsa and also icecast output).

Typical Setup

For most people, they only need one instance of MPD running and have it configured so that it plays songs directly to their computer speakers. So their MPD configuration file will look something like this (comment lines removed to save space):

music_directory “/home/shinobu/.mpd-untracked/music”
playlist_directory “/home/shinobu/.mpd-untracked/playlist”
db_file “/home/shinobu/.mpd-untracked/mpd.db”
log_file “/home/shinobu/.mpd-untracked/mpd.log”
pid_file “/home/shinobu/.mpd-untracked/mpd.pid”
state_file “/home/shinobu/.mpd-untracked/mpdstate”
sticker_file “/home/shinobu/.mpd-untracked/sticker.sql”
user “shinobu”
bind_to_address     “192.168.0.110” # bind it to this computer’s IP address if someone on the LAN wants to use this mpd
port                “6600”
save_absolute_paths_in_playlists “yes”
follow_outside_symlinks “yes”
follow_inside_symlinks “yes”
audio_output {
type            “alsa”
name            “Exelion ALSA output”
format          “44100:16:2”    # optional
mixer_device    “default”
mixer_control   “PCM”
}
replaygain          “track”
volume_normalization “yes”
filesystem_charset      “UTF-8”
id3v1_encoding          “ISO-8859-1”

It’s very straightforward. All of the essential files and folders are customized and defined in the config file. (My mpd directory is .mpd-untracked, since these files/folders are not tracked by my git folder containing all of my dotfiles. See my previous two posts for more info on that setup.) The config file itself resides at /home/shinobu/syscfg/mpd/cfg-alsa — a customized path and filename, which is referenced each time MPD starts on bootup (in my .xinitrc file), with the command “mpd /home/shinobu/syscfg/mpd/cfg-alsa”. MPD will daemonize itself when it is called, so calling it from your .xinitrc file without the “&” argument at the end of the line is OK. Notice how the above configuration avoids running into permission problems, since everything resides under a normal user’s directory.

I then start up ncmpcpp, which by default looks to the mpd host as localhost and port as 6600 — matching the (default) values in my MPD config file above. And it all works rather beautifully.

Multi-MPD Setup for Music Sharing

Back when I still used iTunes (about 3 years ago), I remember it had this neat feature to share your music with anyone on your local network. You can do the same thing with MPD. The trick is to add a separate MPD instance for this feature. I myself have this multi-MPD setup on my desktop, with one MPD for alsa-only output (as seen above), and a second MPD for icecast-only output. Icecast, by the way, is a more general solution not wholly related to MPD for sharing music over the internet — we can use icecast since it plays rather well with MPD (UNIX philosophy FTW!). Since my desktop has a second MPD instance so that it works with icecast, my laptop can now take advantage of the huge music collection that I have on it, remotely (well, as long as I’m on the home network in my case).

Here’s my MPD config file for icecast output:

music_directory “/home/shinobu/.mpd-untracked/music”
playlist_directory “/home/shinobu/.mpd-untracked/playlist”
db_file “/home/shinobu/.mpd-untracked/mpd.db”
log_file “/home/shinobu/.mpd-untracked/mpd-icecast.log”
pid_file “/home/shinobu/.mpd-untracked/mpd-icecast.pid”
state_file “/home/shinobu/.mpd-untracked/mpdstate-icecast”
sticker_file “/home/shinobu/.mpd-untracked/sticker.sql”
user “shinobu”
bind_to_address     “192.168.0.110” # bind it to this computer’s IP address if someone on the LAN wants to use this mpd
port                “6601”
save_absolute_paths_in_playlists “yes”
follow_outside_symlinks “yes”
follow_inside_symlinks “yes”
audio_output {
type            “shout”
name            “exelion HD mpd stream”
encoding        “ogg”           # optional
host            “localhost”
port            “8000”
mount           “/mpd.ogg” # i.e., “http://192.168.0.110/mpd.ogg” is the live stream
# (provided that MPD is actually playing a song)
#————————————————————————————————–#
# See ‘source password’ in /etc/icecast.xml for password                                           #
#————————————————————————————————–#
password        “hackme”
#————————————————————————————————–#
# Use ‘bitrate’ or ‘quality’. Use ‘bitrate’ for static bitrate, or ‘quality’ for variable.         #
# Use one or the other — not both!                                                                #
# bitrate                     “128”                                                                #
#————————————————————————————————–#
quality         “10”
#————————————————————————————————–#
# ‘format’ is the audio format of the stream. The first number is the sample rate in Hertz (Hz);   #
# 44100 Hz is equal to cd quality. The second number is the number of bits per sample; again 16    #
# bits is the quality used on cd’s. The third number is the number of channels; 1 channel is       #
# mono, 2 channels is stereo.                                                                      #
#————————————————————————————————–#
format          “44100:16:2”
protocol        “icecast2”      # optional
user            “source”        # optional
#description        “My Stream Description” # optional
#genre          “jazz”          # optional
#public         “no”            # optional
#timeout            “2”         # optional
}
replaygain          “track”
volume_normalization “yes”
filesystem_charset      “UTF-8”
id3v1_encoding          “ISO-8859-1”

As you can see, much of the configuration is exactly the same as my first MPD config file. You’d want it like this, since you’d want it to use the same music collection and list of playlists. (Well, if you had a private stash of music you didn’t want anyone in your LAN to know about, you’d point to a different music folder.) So the music directory, playlist directory, and database file are all the same. However, you’d want a different set of logfiles for this second MPD instance, so those values are changed. The same goes to the pid and state files. The port number is 6601, so that it doesn’t conflict with my first MPD. The final difference is the audio output, which is set to “shout” and not “alsa”. The “shout” output works with icecast, and the values in here, like port 8000 and “hackme” as the password, are taken straight from the default icecast intallation’s xml file (/etc/icecast.xml — of course, you can call icecast with a different config file location than the default one).

It’s important that you make this second MPD output solely to icecast. If you put in alsa output here, you’ll end up with two different songs playing simultaneously on your desktop’s speakers! (Aside: MPD is perfectly capable of outputting to multiple audio outputs, so you could, if you wanted, make one MPD instance output to both alsa and icecast — suitable if you’re the only one at home and you want your music to “follow” you to all the computers connected to the LAN.) As for icecast itself, you can also call it from your .xinitrc file, after your mpd-calling lines, like so: icecast -b -c /home/shinobu/syscfg/icecast/cfg.xml. The -b flag daemonizes icecast (required since you’re calling it from your .xinitrc file), and the -c flag merely points to the correct config file location. My icecast config file looks like this:

<icecast>
<limits>
<clients>100</clients>
<sources>2</sources>
<threadpool>5</threadpool>
<queue-size>524288</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
<burst-on-connect>1</burst-on-connect>
<burst-size>65535</burst-size>
</limits>

<authentication>
<source-password>hackme(~</source-password>
<relay-password>hackme(~</relay-password>
<admin-user>admin</admin-user>
<admin-password>hackme(~</admin-password>
</authentication>
<hostname>localhost</hostname>
<listen-socket>
<port>8000</port>
</listen-socket>
<fileserve>1</fileserve>

<paths>
<basedir>/usr/share/icecast</basedir>
<logdir>/home/shinobu/.icecast-untracked/log</logdir>
<webroot>/home/shinobu/.icecast-untracked/icecast/web</webroot>
<adminroot>/home/shinobu/.icecast-untracked/icecast/admin</adminroot>
<alias source=”/” dest=”/status.xsl”/>
</paths>

<logging>
<accesslog>access.log</accesslog>
<errorlog>error.log</errorlog>
<loglevel>1</loglevel>
<logsize>10000</logsize>
</logging>
<security>
<chroot>0</chroot>
</security>
</icecast>

Here, the value for the <source-password> tag must match the password specified in your icecast-MPD’s config file. I’ve left it to the default “hackme” here for demonstration purposes — you should change it to something more secure (better safe than sorry!). My password is like @#$k23lsdf9. Make sure the directory in <logdir> exists. As for the <webroot> and <adminroot> directories, copy them straight out of /usr/share/icecast/web and /usr/share/icecast/admin, and place them into your custom location. This will take care of those annoying permission errors that users often face.

Now, on your remote computer, such as your laptop, you need (1) an MPD client, and (2) a music player, like mplayer, to intercept the raw .ogg music stream from your desktop with the icecast-MPD setup. In my ncmpcpp config file on my laptop, I have custom values for the mpd_host and mpd_port fields to be my desktop’s IP address 192.168.0.110 and 6601, respectively (matching the values in the second MPD config file above). So when I fire up ncmpcpp on my laptop, it connects over the LAN network into the music collection residing in my desktop, all with the magic of MPD. Once I activate the ogg stream by playing a file, I can then access this stream with mplayer — specifically, with mplayer -ao alsa -softvol -volume 10 -volstep 1 -prefer-ipv4 http://192.168.0.110:8000/mpd.ogg. I have this long command aliased, of course, to a hotkey in my XMonad configuration file. And within a couple seconds, I get a top-quality, ~500 kbps ogg stream right on my laptop (the quality setting in my MPD config file, set to “10″, is the maximum quality supported by the ogg format; 500 kbps, or a little over 60 KBps, should be nothing on your home ethernet LAN).

So here’s what’s happening with this second MPD instance on my desktop, configured with icecast output: MPD keeps tabs on what my music collection consists of from port 6601, and “plays” to port 8000 — converting my current mp3/flac file into an ogg stream, which is then served by icecast. From my laptop, I then connect to this MPD by the desktop’s IP address and MPD port of 6601 with ncmpcpp. Then, once I tell MPD to start playing, I can then access the ogg stream with mplayer on my laptop. Mplayer simply looks to http://192.168.0.110:8000/mpd.ogg. When icecast on my desktop receives this request from my laptop’s mplayer, it gives the go-ahead and essentially acts as a bridge between my desktop’s icecast-output MPD and my laptop’s mplayer. Mplayer then takes this ogg stream, and outputs it onto my laptop’s speakers using the alsa drivers.

Here’s a picture to describe what’s going on. (The latest LibreOffice Drawing app is really a pleasure to use!)

So I hope this post proves useful to people who are new to Linux and are struggling to get MPD (and especially with Icecast) to work.

EDIT July 17, 2010: Prettier/simpler graphic uploaded.

UPDATE April 7, 2011: Cleaner source code format; fixed some typos; updated graphic.

UPDATE November 1, 2011: Grammar fixes and some rewordings. I’m delighted to say that even after 2 years of using MPD, I still use the setup described in this post. Talk about configuration stability!

Archived from the following link on 7/19/2012.