Best practices for writing a TeamBeam upload client
Abstract
This article will give you information about writing a client software which allows for transferring files via the TeamBeam service.
General information
TeamBeam clients may be written in any programming language as long as it supports communication to the TeamBeam server via the RESTful API which is provided by the TeamBeam backend service. Usage of the API is open to anyone who has a TeamBeam user account.
The API is documented here: API V1 Reference documentation.
Authentication
Users of your upload client need to authenticate themselves before uploading is possible. The login request will provide your client with information about the limitations and allowed features for the logged in user. The server will also provide a session key in form of a Set-Cookie header. Your client should implement session handling by sending the cookie back to the server with every API request. Please note that this is neccessary since the RESTful API works state-less.
Please note
Since http clients only send back session cookies to the originating host, it is important to login using your user's assigned hostname.
If you are writing a generic client which is not bound to a specific TeamBeam storagehost, you may need to allow login against an arbitrary hostname. This will fail with 401 unauthorized, but the HTTP response will contain a Link header of rel-type alternative hinting to the correct storagehost of the user. Your client must then re-authenticate against the correct storagehost, and also use the correct storagehost for all subsequent API requests.
Auto Login
Users may choose to be "kept logged in", as this is known from many web services. If you want to allow this but are unwilling or unable to save a password within the client you have the option to use the auto login feature.
Your client receives an an autologin-cookie, if so requested while authenticating ("autologin": true). This cookie contains a key for exactly one authentication process using the autologin API call:
Open authenticated myTeamBeam web view
If your client shall support forwarding the authenticated user to myTeamBeam without having him to re-authenticate you may use a special URL as a HTTP GET request:
https://{hostname}/my/auth/redirect?k={oneTimeKey}&e={email}
- {hostname} - the TeamBeam server's host name
- {oneTimeKey} - a oneTimeKey, which authenticates the user. See Onetime Key API Documentation
- {email} - the url-encoded e-mail address of the user who shall be authenticated. URL-encoding is important!
Upload process
Sending a TeamBeam transfer is done in three steps:
1. Create a Reservation
This step is neccessary to tell the server that your client is about to upload files and requests them to be stored with a certain set of options. The server will answer by either allowing or denying the request. When the answer is positive, the server provides you with a token which authorizes uploading the files (step 2) and objectIds to use for each file as upload target.
Create Reservation API Documentation
2. Upload files
Now the actual upload takes place. Your client may uploads each file (in chunks or in total) consecutively to the server. It is important that your client sends each file to the resource target by using the correct objectId which has been provided by the server in step 1. Also the provided token must be passed along in form of a http header.
3. Confirm Reservation
Confirming the reservation tells the server that the client is done with uploading. Once again the provided token must be used to confirm the authenticity of the request.
Confirm reservation API Documentation
Upload-Resume
It is good practice to support the resuming of broken uploads. Whenever a upload process of a file unexpectedly stops, your client may try to resume the upload by doing a partial upload (using the Content-Range header) starting at the position whithin the file which has not yet been uploaded. To find out how many bytes of your file have already been uploaded, you may use a HEAD request:
Check uploaded Size API Documentation
Chunked uploads
The TeamBeam backend is a highly scaleable and redundant system which allows for taking certain components offline without interrupting the general service. In order to make it possible to quickly deactivate a component we ask you to split files into chunks when uploading them (using the Content-Range header) rather than uploading a very large file in one piece. Currently this is not required but such a requirement may come in the future, so your code should be already be supporting chunked uploads. As a rule of thumb the following chunk sizes make sense:
- 100 MB for desktop apps
- 10 MB for mobile clients
Session renewal
The TeamBeam backend service may be configured for maximum security by dropping active sessions in a relatively short time span. So your client must be prepared for re-auhtentication if a request is denied due to a 401 unauthorized answer. There are two ways to achieve this:
- cache the user's login credentials and relogin if session times out
- Request a one-time-key immediately after the user logs in as a backup token to use for re-authenticating
If a request is denied because of missing or invalid authentication, it may be retried after re-authentication.
Error handling
Whenever a API request is denied by the server it will answer with a HTTP code that is not in the 2XX range. When reading the API documentation you will find information about specific error codes and their meaning. Please make sure, that your client reacts accordingly. Apart from the specific error codes there are others which can accour anytime:
- 401 unauthorized (user not logged in or session timed out)
- 403 forbidden (your client is doing something illegal. In this case the HTTP body contains a JSON object with detailed information)
- 404 not found (API resource not available, check API documentation)
- 503 maintenance mode (TeamBeam backend is currently unavailable)
- 509 rate limit exceeded (Your clients sends too many requests in a too short time frame)
Error Handling API Documentation
Pseudo code example
This pseudo code shows the recommended process for uploading a single file including chunking and retry-handling.
int retries = 10 //Number of retries in the event of upload failures
long chunkSize = 50*1000*1000 //Initial chunk size of 50 MB. The chunk size is later adapted to fit chunkSeconds
int chunkSeconds = 60 //Uploading of a chunk shall take aprox. 60 seconds
int sleepSeconds = 10 //After network failure sleep a couple of seconds before retrying
File fileToUpload = new File("/path/to/my/file")
try {
boolean success = uploadFile(fileToUpload, 0, false)
} catch (PermanentException e) {
// handle the catastrophe ;-)
}
boolean uploadFile(File file, long startByte, boolean doHead) {
if(doHead) {
//check file size already on the server
try {
startByte = headRequest() //the actual HTTP HEAD request. See https://dev.skalio.net/teambeam/api/v1/#upload-resource-upload-head
} catch (Exception e) {
//an error occurred while checking uploaded file UploadSize
if(e == NotFoundException) {
// 404 Not Found means: There are no bytes on the server yet
startByte = 0
} else {
// some other error when checking upload size
if(retries > 0) {
retries--
sleep(sleepSeconds) // it makes sense to wait a couple of seconds before retrying for the network connection to return
return uploadFile(file, 0, true) //recurse because HEAD failed
} else {
throw PermanentException() //upload failed because max. retries reached
}
}
}
}
//calculate chunk end
long endByte = startByte + chunkSize;
if (endByte > file.size()) {
endByte = file.size(); //the chunk shall not be larger than the remainder of the file
}
//let's upload a chunk of the file
try {
DateTime startDateTime = now() //record start time
long bytesOnServer = putRequest(file, startByte, endByte) //the actual HTTP request. See https://dev.skalio.net/teambeam/api/v1/#upload-resource-upload-put
//upload of chunk was successful
//calculate next chunk size to fit desired time a chunk upload should take
long seconds = now().secondsSinceEpoch() - startDate.secondsSinceEpoch()
double chunkfactor = seconds / chunkSeconds
chunkSize = (chunkSize / chunkfactor).round()
retries = 10; //reset retry counter
if(bytesOnServer < file.size()) {
//this was not yet the last chunk
return uploadFile(file, bytesOnServer, false) //recurse for next chunk
}
} catch (Exception e) {
//an error occurred while uploading chunk
if(retries > 0) {
retries--
sleep(sleepSeconds) // it makes sense to wait a couple of seconds before retrying for the network connection to return
return uploadFile(file, 0, true) //recurse because PUT failed
} else {
throw PermanentException() //upload failed because max. retries reached
}
}
return true; //successful exit after uploading the complete file
}