Debian Planet










Welcome to Debian Planet

Search

All your woody are (not quite, but very very very soon) belong to us.
Main Menu

  • Home

  • Topics

  • Web Links

  • Your Account

  • Submit News

  • Stats

  • Top 10

  • Debian

    These are important Debian sites one should not be without!

  • Official Debian site

  • Package search

  • Mailing list archives

  • Bug reports

  • Debian on CD

  • Unofficial woody CD ISOs

  • Unofficial APT sources

  • Developers' Corner

    Other great Debian news sources:

  • Debian Weekly News

  • Kernel Cousin Debian

    (Debian mailing lists digested)
  • Community Groups

    Need help? You're not alone on this planet.

  • debianHELP

    (User support site)

  • Debian International

  • DebianForum.de

    (Deutsch)

  • EsDebian

    (español)

  • DebianWorld

    (français)

  • MaximumDebian

    (Italiano)

  • DebianUsers

    (Korean)

  • Debian-BR

    (Português)

  • IRC

    The place to get help on a Debian problem (after reading docs) or to just chat and chill is #debian on irc.debian.org.

    Many of the Debian Planet staff live there so pop by and say hello.

    Wanna write?

    Got that latest or greatest scoop? Perhaps you have some important news for the Debian community? Submit a news item!

    Or perhaps you've written a rather ground breaking insight into some aspect of Debian and you feel compelled to share it with others? Knock up a longer editorial article and send it to the editors.

    Sponsorship

    DP is sponsored by Xinit Systems and kieser.net.

    Domains paid for and hosted by uklinux.net.

    Buy your Debian merchandise at DebianShop.com.

    Who's Online

    There are currently, 44 guest(s) and 4 member(s) that are online.

    You are Anonymous user. You can register for free by clicking here.

      
    Master-Slave Package management
    Contributed by Anonymous on Saturday, July 07 @ 13:43:30 BST

    Package Management
    I'm writing this following an install of sid a onto an old 486 Notebook with 8 Megs. As the package - Database grows and grows, the amount of Memory needed to handle it also gets larger. The unstable package database doesn't fit into the measly ram of this machine. It needed to swap continuously.
    A single dpkg-call (independent on whether to list, install or remove one or more Packages) took about 30 minutes, However I have an ingenious idea for distributed package management.


    rob: Why not use potato or slink? With smaller package lists, surely you dont need sid's features on this laptop.

    Here is the idea that came to me:
    It is very often that we come into a situation, that a machine is overloaded with the task of handling it's own
    package-database (take things like

    • the old 486 firewalling you from the nasty big internet
    • the little machine feeding your laser-printer
    • the old Dec-Station you use as X-Terminal
    • the second partition you keep on your Disk to create boot-CD's
    • the embedded-machine to control your washing machine
    • and last (but surely not least) your PDA ).

    So why not distribute the Truck-load of work to machine that would do this in spare-time, like your Desktop >GHzCpu equipped with 100Megs of ram?

    There would have to be something like a net-shared Filesystem on the Slave-machine to be hooked into the Master-machine, and sort of a Deamon
    executing the post-install scripts and so on. (ssh?)

    And now to the Boot - CD-Thing: what about a Plex-86 running the target-installation in non-X mode?

     
    Related Links

  • Comparison by Joey Hess
  • More about Package Management
  • News by rob

    Most read story about Package Management:
    What are the *real* .deb and .rpm differences

    Last news about Package Management:

    Printer Friendly Page  Send this Story to a Friend
  • "Master-Slave Package management" | Login/Create Account | 14 comments
    Threshold


    The comments are owned by the poster. We aren't responsible for their content.

    Re: Master-Slave Package management (Score: 1)
    by crazney on Saturday, July 07 @ 15:24:17 BST
    (User Info)

    or simply, with the price of ram and cd's these days, upgrade?

    [ Reply ]


    s/cd's/cpu's/ (Score: 1)
    by crazney on Saturday, July 07 @ 15:24:45 BST
    (User Info)

    [ Reply ]


    Re: Master-Slave Package management (Score: 0)
    by Anonymous on Monday, July 09 @ 10:58:19 BST

    The poor device doesn't own a CD... more Ram would come at ~150$ at Kingstons...

    A new one would be cheaper...

    [ Reply ]


    Why not use SQL (Score: 1)
    by zoltan on Saturday, July 07 @ 16:54:14 BST
    (User Info)

    As the database grows larger, the flatfile type database isn't gonna work as good anymore. The debian developers should think about using SQL, maybe writing a basic version of a sql daemon only for the pkg database, it would probably improve this situations for the best.

    - Zoltan

    [ Reply ]


    Re: Why not use SQL (Score: 1)
    by crazney on Saturday, July 07 @ 16:59:03 BST
    (User Info)

    haha

    "apt-get install mysql"

    error: apt: cannot find mysql required for command 'install'

    🙂

    [ Reply ]


    Re: Why not use SQL (Score: 1)
    by zoltan on Saturday, July 07 @ 17:10:09 BST
    (User Info)

    mysql is a bit bloated for such a small task like this, but if you insist, try mysql-server

    [ Reply ]


    Re: Why not use SQL (Score: 0)
    by Anonymous on Saturday, July 07 @ 18:05:39 BST

    An intelligently written database that works on files would work just as well (there are some). I.E. instead of loading all the data into memory, search thru the file for what one is looking for, then just return that data. A more intelligently indexed file would also make it a lot easier.

    Plus everyone should go out and buy RAM right now, I can get 256 MB of DDR RAM for my Athlon system at $56, and I just put together a damn-near top-of-the-line system (crappy on-board video card, but I'm not using video anyway) for about $320.

    People have to expect that newer features and functionality will require more hardware - there is a tradeoff.

    [ Reply ]


    Re: Why not use SQL (Score: 0)
    by Anonymous on Sunday, July 08 @ 07:12:24 BST

    (Not the original poster)

    First, every motherboard has a limit on how much memory you can install. So at some point your "solution" degenerates into "everyone should go out and buy a new computer now". (I am close to my limit with 96M - with the latest addition forced by Mozilla and kernel 2.4. I won't even _try_ to run dselect on woody's pool).

    Second, what if it's not "newer features and functionality" that people expect? What if they just need a few upgraded packages for bug fixes? They shouldn't be forced to suffer because the system doesn't scale well.

    Third, and most important, the argument "hardware is cheap, so let's be lazy and write bloated software" is fallacious to the core. Hardware is cheap _BECAUSE_ there is a huge market of bloated software users, and because it's easy for the hardware makers to push their costs down almost at will thanks to "free" trade.

    Sorry to pick on just the second part of your post, I can very much agree with the first part (keep package database as a text file, just add indexing).

    [ Reply ]


    SQL is overkill (Score: 1, Informative)
    by Anonymous on Sunday, July 08 @ 00:10:01 BST

    SQL isn't particularly fast. It's nice and general, and if you are doing very general queries an existing SQL solution might be a good one. I don't think this is the case for Debian. There are a small number of fields, and only a few of those need to be indexed.

    The hash-based GNU db (gdb) and Berkeley DB (libdb?) are both much more appropriate. They are small, they have few requirements (necessary for such a basic tool), and they are really very fast. Faster than a SQL database for the operations Debian would use. It seems like such an easy and obvious choice, I'm really quite surprised Debian isn't using one of these.

    [ Reply ]


    one easy solution: dpkg --root and nfs (Score: 1)
    by joeyh on Saturday, July 07 @ 23:20:49 BST
    (User Info) http://kitenet.net/~joey/

    If you are in an environment where you can safely use nfs without root

    squashing, mount the little slow machine's whole filesystem via nfs on a

    big fat machine, then you can use dselect --root and dpkg --root to make

    dpkg use the nfs mounted filesystem. I've done this, and it works fine.

    You can probably pass apt some more complicated options to make it do the

    same thing.

    A similar approach is to mount it over nfs and then choot into the nfs

    mount and use apt.

    [ Reply ]


    NFS install from the laptop? (Score: 1)
    by jope on Sunday, July 08 @ 19:24:48 BST
    (User Info)

    Is there any way to kick off this kind of NFS/chroot install from the laptop though,
    rather than from the beefy system?

    Before answering the above, there is a prerequisite: Does the problem described here bite early enough, that just getting to the point of NFS exporting the laptop's filesystem would be painful?

    I don't recall the option during any of my base installs to NFS export my filesystem, only the option to NFS mount another. At very least, that much needs to be possible during the base installation.

    [ Reply ]


    Re: Master-Slave Package management (Score: 1, Interesting)
    by Anonymous on Monday, July 09 @ 13:45:36 BST

    The solution to this problem will address the problem of maintaining a cluster of nodes, as well as dispersed but functionally equivalent (and equivalently configured) sealed servers on a wide area network, whilst also presenting a solution to the problems unique to administering roaming nodes.

    I've often wondered what, if any, impediments there are to the idea of having the Packages database as well as the debconf databases accessed through LDAP rather than through the local file store, with state data keyed both by a user-assigned token (nodename) and an administratively assigned token (nodetype).

    This problem will need to be solved in the near future; the first distribution to get the "right" approach here will also be the first distribution to make it possible to detach/reconnect any given node from an amorphous data "cloud", make it possible to treat nodes as functional units, and make it possible to minimise data redundancy between cluster members.

    The underlying technology for building storage networks is here; Gigabit Ethernet, iSCSI, FCAL, and GFS give you all the infrastructure components required to build a network supporting clustered, distributed, or roaming nodes.

    Food for thought, indeed.

    -- why doesn't *your* package have md5sums?

    [ Reply ]


    non-x86 ports (Score: 0)
    by Anonymous on Tuesday, July 10 @ 15:46:01 BST

    I run sid on an old nubus powermac and *man* is that thing slow. I think ports to older architectures that are just now getting good infrastructure support in glibc, Linux kernel, etc. could really use a lighter implementation. The master/slave node idea could be a good option for these types of users.

    Potato just doesn't cut it.

    -l

    [ Reply ]


    apt is broken and needs to get fixed (Score: 0)
    by Anonymous on Saturday, July 14 @ 19:09:07 BST

    There is no reason why "apt" or "dpkg" need to consume lots of memory. The complete "unstable" apt package info without descriptions is about 2.3M, even without doing anything special. Even that doesn't need to be all read into memory. "apt" is broken and it should just get fixed to use less memory.

    [ Reply ]


    Based on: PHP-Nuke

    All logos and trademarks in this site are property of their respective owner. The comments are property of their posters, all the rest © 2000 by Debian Planet

    You can syndicate our news using the file backend.php.