Skip to content

create new fpb_env package with fixed dependencies #704

Open

Description

@gdementen

We have had too many problems of too old or too recent dependencies

Activity

  1. added this to the 0.30 milestone on Oct 11, 2018
  2. modified the milestones: 0.30, 0.31 on Jan 31, 2019
  3. alixdamman commented on May 28, 2019

    @alixdamman
    Collaborator

    See also #767

  4. added a commit that references this issue on May 28, 2019
    664572e
  5. changed the title [-]fix dependencies in larrayenv[/-] [+]create new fpb_env package with fixed dependencies[/+] on May 29, 2019
  6. alixdamman commented on May 29, 2019

    @alixdamman
    Collaborator

    @gdementen
    Question 1: Do you want to remove larrayenv?
    Question 2: fpb_env should be a package or a metapackage?

  7. gdementen commented on May 29, 2019

    @gdementen
    ContributorAuthor
    1. Not at this point as it is useful for people outside the fpb (eg croatians). If its maintenance takes too much time, we can always re-evaluate that later.
    2. Unsure. I guess a package, so that we can provide a Windows menu to upgrade it, or is that possible with a metapackage too? On the long run, I suppose a package will be better anyway because I guess that at some point we will want to add FPB-specific pieces of code there that are not large enough to warrant creating a separate package for them.
  8. removed this from the 0.31 milestone on Aug 1, 2019
  9. added this to the 0.32 milestone on Aug 8, 2019
  10. removed this from the 0.32 milestone on Sep 24, 2019
  11. 2 remaining items

  12. added this to the 0.33 milestone on Dec 11, 2020
  13. modified the milestones: 0.33, 0.34 on Aug 17, 2021
  14. self-assigned this
    on Sep 20, 2022
  15. added
    priority: BLOCKERcannot make a release without these issues resolved
    and removed on Sep 20, 2022
  16. modified the milestones: 0.34, 0.34.1 on Feb 14, 2023
  17. modified the milestones: 0.34.1, 0.35 on Oct 25, 2023
  18. gdementen commented on Oct 25, 2023

    @gdementen
    ContributorAuthor

    This issue is really a pain and I am a bit stuck with it.

    On one hand, not having a solution for this keeps being a problem over and over. New installs, or users installing another package and getting updated dependencies which conflicts with LArray in some way (usually via new warnings but sometimes with a crash).

    On the other hand our users also need to install miscellany packages on top of larrayenv (scipy, pyarrow, rpy2, statsmodels, sympy, yaml, ...) and having very strict version requirements would in some cases make installing those packages impossible. So one solution would be to include all those packages in our mega-package... but some of those packages do not exist in anaconda or are so outdated they are inusable. The solution to that is to switch to either conda-forge (and keep using conda) or to pip.

    • pip usually has all packages and all their versions.
    • conda-forge currently has a lot more packages than anaconda, and they are relatively quickly updated.

    The problem with both is the flip-side of the advantage of being community-maintained: there is no gatekeeping by Anaconda Inc => there is a higher risk -- at least the way I see it -- of a compromised package getting through either directly when users install more packages (hard because of the strict dependencies versions but not impossible) or, in our machine, most likely via one of our indirect dependencies when we produce updated packages.

    One solution to the "users installing new packages" problem is to prevent them from doing so and produce a new version of our mega package with ever more packages each time one user needs an extra package. One option to do this would be to create offline installers containing all packages using conda constructor (https://github.com/conda/constructor - https://conda.github.io/constructor/howto/).

    In the pip world, I think our best bet would be to run our own private pypi repository (https://packaging.python.org/en/latest/guides/hosting-your-own-index/). Unsure how hard/how much effort it would mean to keep that running.

  19. modified the milestones: 0.35, 0.36 on Oct 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions