python - Pytorch: copy.deepcopy vs torch.tensor.contiguous()?

Question

In python torch, it seems copy.deepcopy method is generally used to create deep-copies of torch tensors instead of creating views of existing tensors. Meanwhile, as far as I understood, the torch.tensor.contiguous() method turns a non-contiguous tensor into a contiguous tensor, or a view into a deeply copied tensor.

Then, do the two code lines below work equivalently if I want to deepcopy src_tensor into dst_tensor?

org_tensor = torch.rand(4)
src_tensor = org_tensor

dst_tensor = copy.deepcopy(src_tensor) # 1
dst_tensor = src_tensor.contiguous() # 2

If the two work equivalent, which method is better in deepcopying tensors?

question from:https://stackoverflow.com/questions/65894586/pytorch-copy-deepcopy-vs-torch-tensor-contiguous

Answer 1

torch.tensor.contiguous() and copy.deepcopy() methods are different. Here's illustration:

>>> x = torch.arange(6).view(2, 3)
>>> x
tensor([[0, 1, 2],
        [3, 4, 5]])
>>> x.stride()
(3, 1)
>>> x.is_contiguous()
True
>>> x = x.t()
>>> x.stride()
(1, 3)
>>> x.is_contiguous()
False
>>> y = x.contiguous()
>>> y.stride()
(2, 1)
>>> y.is_contiguous()
True
>>> z = copy.deepcopy(x)
>>> z.stride()
(1, 3)
>>> z.is_contiguous()
False
>>>

Here we can easily see that .contiguous() method created contiguous tensor from non-contiguous tensor while deepcopy method just copied the data without converting it to contiguous tensor.

One more thing contiguous creates new tensor only if old tensor is non-contiguous while deepcopy always creates new tensor.

>>> x = torch.arange(10).view(2, 5)
>>> x.is_contiguous()
True
>>> y = x.contiguous()
>>> z = copy.deepcopy(x)
>>> id(x)
2891710987432
>>> id(y)
2891710987432
>>> id(z)
2891710987720

contiguous()

Use this method to convert non-contiguous tensors to contiguous tensors.

deepcopy()

Use this to copy nn.Module i.e. mostly neural network objects not tensors.

clone()

Use this method to copy tensors.