Repository navigation
implement Frame #81
Description
which would store different fields/columns with (potentially) different types.
Each field from an LFrame should also be accessible via __getattr__.
We might want to use a ColumnArray instead of a numpy structured array like in LIAM2 as the storage backend so that we can add fields efficiently.
See the https://github.com/liam2/larray/tree/structured_array branch.
Activity
gdementen commentedon Apr 26, 2017
There are several big issues to resolve:
A: I think they should return an LFrame by default. API-wise, I think it would work, but it would be a performance hit, relative to what we have now, because the LFrame would at best need to be converted to LArray at the first sign of an aggregate.
A: the easiest would be to raise an Exception in that case, but then we cannot return an LFrame by default for read_csv. One option would be to transparently convert to LArray when doing aggregates. This screams pandas.NDFrame all over again but I think it is sane.
A: I do not think we can avoid this, but we do shouldn't need to re-implement user-facing methods, but rather make ColumnArray implement the subset of the numpy API that we use internally. This is some significant work, but should not be too hard.
We should see what xarray does here.
[-]implement LFrame[/-][+]implement Frame[/+]alixdamman commentedon Aug 17, 2018
@gdementen why?
gdementen commentedon Aug 19, 2018
because currently read_csv returns LArrays where aggregates on the column axis work. This would be a massive backward incompatibility/pain in the * for our users if suddenly all their data loaded from .csv files failed to aggregate over the column axis.