| 00:21.46 | *** join/#maemo infobot (ibot@rikers.org) |
| 00:21.46 | *** topic/#maemo is Welcome to #maemo http://maemo.org/intro/ | N9/harmattan related questions please in #harmattan ! | "firmware" see ~lf (or ~tabletsdev ~combined ~emmc ~flasher) sdk: ~sb | http://wiki.maemo.org/Community_Council | Source: http://wiki.maemo.org/Sources | Chanlog: http://mg.pov.lt/maemo-irclog | #maemo-ssu is where the (few) devels and maintainers of the maemo-future meet |
| 00:21.46 | *** mode/#maemo [+v infobot] by ChanServ |
| 01:21.17 | *** join/#maemo LauRoman|Alt (~LauRoman@5-14-11-114.residential.rdsnet.ro) |
| 01:31.18 | *** join/#maemo luke-jr (~luke-jr@unaffiliated/luke-jr) |
| 02:38.21 | *** join/#maemo vectis_ (~vectis@78.141.51.129) |
| 03:06.54 | timeless | DocScrutinizer: yes. |
| 03:07.51 | *** join/#maemo tm (~sinnlos@unaffiliated/tm) |
| 03:10.10 | timeless | Unfortunately, this is a basic flaw in the way Debian packages work. The binaries don't include enough information to capture how they were built |
| 03:12.52 | timeless | Romoxa (Oleg Romashin) most of the build optimizations and continued working on fennec for a while |
| 03:33.21 | *** join/#maemo vahe (~vahe@unaffiliated/vahe) |
| 04:02.48 | *** join/#maemo trx (ns-team@cable-188-2-25-100.dynamic.sbb.rs) |
| 04:02.48 | *** join/#maemo trx (ns-team@devbin/founder/trx) |
| 04:27.21 | *** join/#maemo sunshavi (~user@181.64.192.240) |
| 04:44.24 | *** join/#maemo luke-jr (~luke-jr@unaffiliated/luke-jr) |
| 05:18.21 | *** join/#maemo Hurrian (~hurrian@unaffiliated/rikanee) |
| 06:05.39 | *** join/#maemo jonwil (~jonwil@27-33-80-219.tpgi.com.au) |
| 07:04.55 | *** join/#maemo danjackson (~user@pa49-199-40-26.pa.vic.optusnet.com.au) |
| 07:13.53 | *** part/#maemo valdyn (~valdyn@host-88-217-143-53.customer.m-online.net) |
| 07:30.20 | *** join/#maemo vahe (~vahe@unaffiliated/vahe) |
| 07:31.14 | *** join/#maemo vahe (~vahe@unaffiliated/vahe) |
| 08:12.13 | *** join/#maemo vahe1 (~vahe@unaffiliated/vahe) |
| 08:13.06 | *** join/#maemo vahe1 (~vahe@unaffiliated/vahe) |
| 08:18.28 | *** join/#maemo xy2_ (~xy2_@94.239.177.175) |
| 08:24.41 | *** join/#maemo florian_kc (~florian_k@Maemo/community/contributor/florian) |
| 08:24.46 | *** join/#maemo phlixi (~phlixi@ppp-46-244-175-84.dynamic.mnet-online.de) |
| 08:29.13 | *** join/#maemo Pali (~pali@Maemo/community/contributor/Pali) |
| 08:29.48 | *** join/#maemo phlixi (~phlixi@ppp-46-244-175-118.dynamic.mnet-online.de) |
| 08:34.10 | enyc | timeless: hrrm... isn't that what '-dbg' and source packages are for? |
| 08:34.34 | timeless | no |
| 08:34.44 | enyc | timeless: reproducible-builds project being helpful, you can check you can rebuild he same checksum binary etc.? |
| 08:34.58 | *** join/#maemo vectis (~vectis@78.141.51.129) |
| 08:35.00 | timeless | they don't provide a way to guarantee a complete build env |
| 08:35.22 | timeless | enyc: the problem is that a binary package doesn't tell you which precise versions of dependent packages |
| 08:35.39 | timeless | it doesn't tell you which version of the compiler was used |
| 08:35.41 | timeless | or the linker |
| 08:35.42 | timeless | or... |
| 08:35.50 | enyc | timeless: aaat the build-log i see |
| 08:35.59 | timeless | yes, if you manage to get matching checksums, you know that you reproduced things |
| 08:36.14 | timeless | but, when you don't (which will happen 99.99% of the time, due to things that embed timecodes) |
| 08:36.19 | timeless | you don't know anything |
| 08:36.24 | enyc | timeless: surely its' expected to use tha same common pbuilder / debian release |
| 08:36.40 | enyc | timeless: thats part of why they use pbuilder, create new chroot with minimum deps for building each time |
| 08:37.14 | enyc | but then i understand it depends what 'updates' to a release were applied before/after a package was built |
| 08:37.36 | enyc | e.g. gcc with retpoline fixes -- has package been rebuilt-or-not, etc etc |
| 08:38.27 | timeless | enyc: anyway, the point is that debian packages do not encode enough info |
| 08:38.51 | timeless | if the package is from along time ago, (5-10 years), you're SOL if the build environment is long gone |
| 08:39.12 | enyc | have you made a propased patch to dpkg/tools etc.?, oloked into options for build logs/info to be stored eithre separately or embedded in the debs in a way that won't affect existing systems installing them...? |
| 08:39.37 | timeless | no |
| 08:39.44 | timeless | for one, i'm not sure it's entirely solvable |
| 08:39.55 | timeless | one could at least encode the version of installed packages |
| 08:40.18 | enyc | the pbuilder creates buildlogs |
| 08:40.29 | DocScrutinizer05 | embeeded timestamps, HAHA great. I guess a VM with patched time_t time(time_t *t) that simply returns a configurable constant might be useful |
| 08:41.09 | timeless | there's a faketime module, i use it to talk to openssl |
| 08:41.11 | enyc | well acutalyl reproducible-builds work to get rid of all build times etc. of course |
| 08:42.29 | timeless | enyc: anyway, build logs that are on a server that don't persist aren't helpful |
| 08:42.33 | Maxdamantus | I think I've noticed that particular packages in nix have workarounds for removing build times from things that store them. |
| 08:42.38 | timeless | remember, DocScrutinizer05 is asking about packages from 5-10 years ago |
| 08:42.53 | DocScrutinizer05 | actually jonwil is |
| 08:42.55 | timeless | the source servers in a number of cases are gone |
| 08:43.03 | timeless | heck, even the build env distros are gone |
| 08:43.04 | Maxdamantus | (and as a standard feature, file times are all set to `0` after building a derivation in the store) |
| 08:43.09 | timeless | (which scratchbox,...) |
| 08:43.27 | enyc | Maxdamantus: thats' to make the build reproducible ? |
| 08:44.07 | Maxdamantus | enyc: not exactly. |
| 08:44.20 | Maxdamantus | enyc: the build should be reproducible enough without that. |
| 08:44.22 | timeless | really, to some extent, anything that isn't part of the shipping .deb is probably a bad thing to rely on |
| 08:44.50 | timeless | it's pretty easy to not end up w/ the associated "dbg" packages in case no one used them and the repo got lost |
| 08:45.12 | enyc | well you say the problem isn't solvab/e, mut didn't rcatch what parts ooyou want solved and why, what the compatibilite problems would be, and how you do intend to solve those that should be solved. |
| 08:45.44 | timeless | anyway,... solvable? maybe, it'd probably require logging all installed package versions and checksums for all files |
| 08:46.16 | timeless | the question is ~ can one rebuild a build env in order to rebuild a package? |
| 08:46.34 | timeless | that's what's desired and in a number of cases desirable |
| 08:46.52 | Maxdamantus | That should be relatively doable in nix. |
| 08:46.54 | timeless | a fun version of this problem: |
| 08:46.57 | enyc | timeless: hrrm like i mentiont to leste people they shouldn't be expecting to reply on current githumb for contributors .. if you want to keep and reprouce/upstream fixes you need things like copyright file and clear status... hey-ho. |
| 08:47.04 | Maxdamantus | (though that's not relevant in this case) |
| 08:47.12 | timeless | a package X depends on a build-dep Y w/ some vague version requirement (not absolute) |
| 08:47.28 | timeless | in one version of Y, it defines a macro w/ one value, in another, it defines it with a different value |
| 08:47.28 | DocScrutinizer05 | could at least from now on make, cmake et al take care to include all info to the .deb per default? |
| 08:47.55 | Maxdamantus | What is "all info"? Every package on the build system? |
| 08:48.08 | Maxdamantus | or every package transitively reachable through the dependencies? |
| 08:48.15 | timeless | the resulting X package will have totally different shapes depending on the define, which is not something that you can tell by looking at the package |
| 08:48.21 | Maxdamantus | or just the dependencies? |
| 08:48.52 | DocScrutinizer05 | ouch |
| 08:49.20 | timeless | Maxdamantus: it would be safer if it did all packages, but for privacy reasons everyone would object to that |
| 08:49.53 | timeless | i think from a reproducible perspective, if your build env was able to record all packages that contributed to files that were opened during the build process |
| 08:49.58 | timeless | you'd get something vaguely good |
| 08:50.12 | DocScrutinizer05 | :nod: |
| 08:50.20 | DocScrutinizer05 | augias task |
| 08:51.11 | timeless | and again, what do you do if something is modifying how you see files |
| 08:51.13 | Maxdamantus | I'm not sure how it normally works on debian-based systems, though I think it hasn't worked this way with maemo, but: if everyone agrees that all official packages are built only by an automatic build server, it should be fairly reproducible. |
| 08:51.28 | timeless | Maxdamantus: i worked on maemo |
| 08:51.37 | timeless | the build servers more or less did things in parallel |
| 08:51.40 | Maxdamantus | since the build maintainers can just keep a git repository with all deb sources. |
| 08:51.52 | timeless | you could have races where different packages were built/used in parallel |
| 08:51.59 | timeless | such things were real problems |
| 08:52.14 | timeless | specifically, i tried to build a source code index of a given binary release |
| 08:52.14 | Maxdamantus | Heh, fun. |
| 08:52.17 | timeless | it wasn't really possible |
| 08:52.31 | timeless | because different binaries used different versions of dependent packages |
| 08:52.32 | DocScrutinizer05 | idly muses about keeping complete VMs in source 'repo' |
| 08:52.49 | DocScrutinizer05 | one for each package - OUUUFF |
| 08:52.50 | timeless | DocScrutinizer05: real professional projects do that, more or less |
| 08:53.06 | timeless | not archiving the build env leads to problems like this |
| 08:53.20 | Maxdamantus | tbh, the complete VMs thing doesn't seem particularly unattainable. |
| 08:53.31 | Maxdamantus | If your storage mechanism involves deduplication, it should be fine. |
| 08:54.05 | Maxdamantus | eg, one obvious way of achieving that would be to fire off every build in an isolated snapshot of the filesystem. |
| 08:54.34 | Maxdamantus | then you just save the filesystem tree as a git tree in a bit repo. |
| 08:54.34 | timeless | Maxdamantus: i used dedup on the source system to avoid blowing up space w/ my weekly index |
| 08:54.43 | timeless | (i used zfs) |
| 08:55.04 | sparre | At one of my customers, we launch a VM on Amazon for every build-and-test run. |
| 08:55.19 | Maxdamantus | Yeah, I was thinking of the "snapshot" part in terms of btrfs, but zfs works too. |
| 08:55.30 | timeless | this was over a decade ago |
| 08:55.33 | DocScrutinizer05 | looked into dedup solutions a lot, for local workstation (btrfs, ext4) - it's a PITA and nothing _really_ worked |
| 08:55.56 | timeless | in my case, since the files were generally predictably named, it was pretty easy |
| 08:56.07 | timeless | i built to a stage, got rid of the package versions from directory names |
| 08:56.15 | timeless | and then rsync'd over the previous week's files |
| 08:56.23 | timeless | and snapshotted |
| 08:56.28 | *** join/#maemo florian_kc (~florian_k@Maemo/community/contributor/florian) |
| 08:56.41 | DocScrutinizer05 | yeah, that's somewhat sustainable and feasible |
| 08:57.06 | timeless | but, that didn't solve the stupid "x uses a@1; y uses a@2; z uses b@2 which uses a@?" |
| 08:57.23 | DocScrutinizer05 | :nod: |
| 08:59.05 | DocScrutinizer05 | the problem however feels like homegrown, the Godfathers of unix had a pretty clear notion about how to tackle this at very least on .SharedObject library level |
| 08:59.35 | timeless | sharedobjects are one things, but it doesn't cover macros |
| 08:59.42 | DocScrutinizer05 | "new" stuff should strictly follow same principles |
| 08:59.59 | DocScrutinizer05 | yeah, sure, that's why >>at least for...<< |
| 09:01.09 | timeless | anyway.... it sucked... and was very frustrating not being able to answer questions about what a binary was thinking |
| 09:01.10 | DocScrutinizer05 | coders would need to learn to *always* think fore/back-ward compatibility |
| 09:02.02 | timeless | grr. cat wants to go out |
| 09:02.30 | DocScrutinizer05 | but yeah, when you make a new (version of) package, you usually don't give a shit about backward and even less forward compatibility |
| 09:18.17 | DocScrutinizer05 | a few days ago looked into snap packages first time. Also sort of PITA, NFC where the apache running in snap env is keeping its vhost config |
| 09:20.38 | DocScrutinizer05 | the damn nextcloud snap package even has a LetsEncrypt running, but I can't find how to enable apache HTTPS |
| 09:23.08 | *** join/#maemo geaaru (~geaaru@151.60.136.99) |
| 09:30.19 | timeless | is mod_md in hat thing? |
| 09:40.17 | *** join/#maemo M4rtinK (~M4rtinK@ip-78-45-137-21.net.upcbroadband.cz) |
| 09:45.33 | jonwil | I think as far as build prerequisites go, the only option is to assume that if "configure" doesn't complain about stuff being out of date, its suitable to use. |
| 09:59.50 | *** join/#maemo Hurrian (~hurrian@unaffiliated/rikanee) |
| 10:09.02 | *** join/#maemo florian_kc (~florian_k@Maemo/community/contributor/florian) |
| 10:38.33 | *** join/#maemo luke-jr (~luke-jr@unaffiliated/luke-jr) |
| 10:39.45 | *** join/#maemo vahe (~vahe@unaffiliated/vahe) |
| 11:40.28 | *** join/#maemo shentey (~shentey@x55b4f555.dyn.telefonica.de) |
| 12:33.34 | *** join/#maemo Vajb (~Vajb@85-76-2-32-nat.elisa-mobile.fi) |
| 13:47.05 | *** join/#maemo luke-jr (~luke-jr@unaffiliated/luke-jr) |
| 14:23.38 | *** join/#maemo florian (~florian_k@Maemo/community/contributor/florian) |
| 14:24.52 | *** join/#maemo KotCzarny (ariel@31.135.195.151) |
| 14:54.20 | *** join/#maemo M4rtinK (~M4rtinK@ip-78-45-137-21.net.upcbroadband.cz) |
| 15:51.29 | *** join/#maemo xes (~xes@unaffiliated/xes) |
| 16:53.07 | *** join/#maemo TheKit (~nekit@178.45.233.188) |
| 17:05.50 | *** join/#maemo xes (~xes@unaffiliated/xes) |
| 17:26.28 | *** join/#maemo xy2_ (~xy2_@94.239.177.175) |
| 20:24.08 | Vajb | hehe interesting read of maemo builds back there |
| 20:24.54 | Vajb | last nite i wanted to prepare build enviroment for my upcoming android course... |
| 20:25.10 | Vajb | can admit that i don't like android studio |
| 20:48.16 | *** join/#maemo Kilroo (~Kilroo@71.71.97.50) |
| 22:31.16 | *** join/#maemo sunshavi (~user@181.64.192.240) |
| 22:45.34 | *** join/#maemo jonwil (~jonwil@27-33-80-219.tpgi.com.au) |