devlog

As we decided on the command line interface, how will you implement the S3 functionality for the common trait?

There are five functions. We assume there will be a URL in the form s3://my-bucket.s3.amazonaws.com/remote-directory/ We will receive all components of this from the user.

Ok, that seems feasible. You didn’t resolve the authentication part but we defer that discussion until later.

I assume the environment contains the keys. We won’t discuss that until we have a working setup. Now, we have init that will create the directory if it doesn’t exist. It will copy the remote guid file. It will register the necessary elements.

What could go wrong?

If the directory exists, it won’t be created. If it has files, though, we should give error I think.

You don’t allow people to move their remotes manually?

There will be --skip-init option to skip the entire init process. It will be used to skip the init when it’s already initialized. I didn’t add it yet though.

Users won’t be able to skip certain parts of the init process?

No, I won’t allow that. Initialization must be atomic. It’s a simple operation actually, create a directory and add a text file into it. There are no fancy parts. I intend to add that option for multi-repo remotes. When a single remote is used for multiple repositories, there is no need to initialize it. GUID will be received from the remote itself.

I think that could be implemented quickly. At the XvcRemote level. It will run receive function instead of send, right?

I think --skip-init cannot be abstracted at the higher level. It will need to be implemented at the per-repo level, in init function. GUID is created there currently, and I don’t think we can have a general function for this.

Receiving GUID file is a download functionality. In place of {REMOTE_PATH}, it will supply .xvc-guid and download the file to a temporary location.

Yes, but in generic case, for example, {REMOTE_PATH} is {REMOTE_CACHE_PATH} and that path includes the {XVC_DIR}. We don’t have generic path requirements to download arbitrary paths.

Maybe you should change that as well. Commands for upload and download should allow arbitrary paths to be downloaded and Xvc should be able to download .xvc-guid .

Currently it uses {RELATIVE_CACHE_PATH} to fill the remote path. You say the upload, download, delete commands in generic remotes should contain {REMOTE_PATH} or {LOCAL_PATH} , and these should be filled with necessary elements, like {XVC_GUID}/{RELATIVE_CACHE_PATH} in runtime. Only those two should be present?

Yes, more or less that’s. I think having more granular commands doesn’t help. rclone copy {LOCAL_PATH} {URL}{REMOTE_DIR}{REMOTE_PATH} should be enough as an upload command. We can determine the what fills {LOCAL_PATH} and {REMOTE_PATH} depending on the use case.

For cache operations, {REMOTE_PATH} will contain {XVC_GUID}/{RELATIVE_CACHE_DIR} but when GUID file is needed we’ll supply .xvc-guid for it.

True. And checking whether the commands contain these two identifier is easier in init. All commands should contain {REMOTE_PATH} and {LOCAL_PATH}. In this case init will not require specialized identifiers.

Yep. Let me check this while reading more about S3.