Okay, but CPU's, GPU's, and ASICs can all be manufactured. And I don't see anyone building a computer whose software can't be copied unless it's for artificial reasons like DRM.
So it seems like the question is whether computers as we know them can do general intelligence at all? If they can, it will be easy to clone.
If they can't, then it would imply that general intelligence is something more than information processing as we know it; perhaps some kind of dualism?
How do you take a subset of knowledge from neural net A such as cats have fir and merge it with neural net B?
It's not a software or hardware problem it's a data problem as it's not obvious what part of neural net A encodes fir and what part encodes cat's and how you map that to B's encoding of fir and cats while connecting them.
Now, AI is not necessarily going to be neural net's, but it's also not necessarily understandable what all the little bit do, just that they work.
That seems like a problem with software composition? If you just want to clone a neural network, you can do that without knowing how it works. In git terms, we have fork but not merge.
So it seems like the question is whether computers as we know them can do general intelligence at all? If they can, it will be easy to clone.
If they can't, then it would imply that general intelligence is something more than information processing as we know it; perhaps some kind of dualism?