11

I am currently trying to save a list of numpy arrays into a single file, an example of such a list can be of the form below

import numpy as np
np_list = []
for i in range(10):
    if i % 2 == 0:
        np_list.append(np.random.randn(64))
    else:
        np_list.append(np.random.randn(32, 64))

I can combine all of them using into a single file using savez by iterating through list but is there any other way? I am trying to save weights returned by the function model.get_weights(), which is a list of ndarray and after retrieving the weights from the saved file I intend to load those weights into another model using model.set_weights(np_list). Therefore the format of the list must remain the same. Let me know if anyone has an elegant way of doing this.

Improve this question
2

1 Answer 1

Reset to default
18

I would go with np.save and np.load because it's platform-independent, faster than savetxt and works with lists of arrays, for example:

import numpy as np

a = [
    np.arange(100),
    np.arange(200)
]
np.save('a.npy', np.array(a, dtype=object), allow_pickle=True)
b = np.load('a.npy', allow_pickle=True)

This is the documentation for np.save and np.load. And in this answer you can find a better discussion How to save and load numpy.array() data properly?

Edit

Like @AlexP mentioned numpy >= v1.24.2 does not support arrays with different sizes and types, so that's why the casting is necessary.

Improve this answer
7
  • This does not work for a list of numpy arrays
    – DVK
    Feb 13, 2020 at 2:09
  • 4
    Careful with this approach. save will do np.array(a) before saving. So b will be an array, not a list. And as others have found out, making an array from a list of arrays has its hidden dangers. Don't do it naively.
    – hpaulj
    Feb 13, 2020 at 2:27
  • 1
    @hpaulj oh, that's True! Thanks for the insight.
    – marcos
    Feb 13, 2020 at 2:28
  • 1
    It does not let me answer, so I go here. It did not worked for me. The solution was to use np.savez. I would do the complete answer if I could comment.
    – J Agustin Barrachina
    Jul 1, 2022 at 13:49
  • 1
    using numpy 1.24.2 and it no longer works if storing arrays of different size or different type. Use instead: np.save('a.npy', np.array(a,dtype=object), allow_pickle=True)
    – AlexP
    Feb 9, 2023 at 2:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.