Is there a way to access the binary file (shell script) for a particular job of a concrete workflow through ASM?
We're looking to store the original binary with the provenance data we're saving for every run of a particular workflow, so that if the binary is updated, we still have access to the original binary used for a particular run.
Thanks in advance. Regards,
Jonas
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Previously I was able to access the workflow from the repository through the filesystem, but as we have since moved to separate guse and portal servers, this is no longer an option.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This feature is not supported in ASM since its concept is based on using well-defined workflows created previously to guarantee that the users won't change essential properties of a workflow. Therefore defining new arcs, or updating binaries are not supported for instance.
Cheers,
akos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm not trying to change the binary, like I wrote in my original post we're simply looking to save a copy of the full binary with the other provenance data we already collect.
Cheers,
Jonas
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One possible workaround is to download files through http.
In the JSDL of a job to be submitted, all the files stored in the storage component is defined as URL, binaries included, so you can use those URLs to make a copy from the binary.
Hope it helps,
cheers,
akos
Last edit: Akos Balasko 2015-02-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Can you point me in the right direction for actually getting at the JSDL data? I've looked at getDesc() on JobPropertyBean (as used in get/setJobAttribute() on ASMWorkflowConfiguration), but that contains only the attributes we have set ourselves.
Thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I just suggested to check a JSDL generated automatically for job submission. (Btw, You must set DCI Bridge into debug mode to keep these files in the bridge's working directory.) So I just said to check the URLs from there, investigate their folder hierarchy and then you can extend you code to download the binaries sidestepping ASM.
Cheers,
akos
Last edit: Akos Balasko 2015-02-19
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
in the JSDL. However, the file execute.bin doesn't exist in that location (and never does, at any point in time, I've checked with inotify). This contrasts with the inputs (Job0/inputs/0/0, etc) which do get written.
Any suggestions at this point?
Thanks. Best regards,
Jonas
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
the URI looks a bit weird, but it shows the meaning. As you see it points to a servlet: "http://192.168.33.11:8080/storage/getFile" and one argument (path)is passed to it to get the file. I think this path is wrong because it does not point to the temp folder in where the real files staying.
So I would suggest to replace the path to the correct one.
And it can be automatized, so you can build this link creating the downloading process into your portlet.
Cheers,
akos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hey,
Is there a way to access the binary file (shell script) for a particular job of a concrete workflow through ASM?
We're looking to store the original binary with the provenance data we're saving for every run of a particular workflow, so that if the binary is updated, we still have access to the original binary used for a particular run.
Thanks in advance. Regards,
Jonas
I'm still looking for an answer to this question.
Previously I was able to access the workflow from the repository through the filesystem, but as we have since moved to separate guse and portal servers, this is no longer an option.
Dear Jonas,
This feature is not supported in ASM since its concept is based on using well-defined workflows created previously to guarantee that the users won't change essential properties of a workflow. Therefore defining new arcs, or updating binaries are not supported for instance.
Cheers,
akos
Hi Akos,
I'm not trying to change the binary, like I wrote in my original post we're simply looking to save a copy of the full binary with the other provenance data we already collect.
Cheers,
Jonas
Hi Jonas,
One possible workaround is to download files through http.
In the JSDL of a job to be submitted, all the files stored in the storage component is defined as URL, binaries included, so you can use those URLs to make a copy from the binary.
Hope it helps,
cheers,
akos
Last edit: Akos Balasko 2015-02-17
If that works, it would be perfect. Will look into that. Thanks!
Hi Akos,
Can you point me in the right direction for actually getting at the JSDL data? I've looked at getDesc() on JobPropertyBean (as used in get/setJobAttribute() on ASMWorkflowConfiguration), but that contains only the attributes we have set ourselves.
Thanks.
Dear Jonas,
I just suggested to check a JSDL generated automatically for job submission. (Btw, You must set DCI Bridge into debug mode to keep these files in the bridge's working directory.) So I just said to check the URLs from there, investigate their folder hierarchy and then you can extend you code to download the binaries sidestepping ASM.
Cheers,
akos
Last edit: Akos Balasko 2015-02-19
Hi Akos,
So I find URI's like this http://192.168.33.11:8080/storage/getFile?path=http:__192.168.33.11:8080_forward-modelling-portlet/10196/qwertyuiop01424859190351_2015-02-25-101314/Job0/execute.bin
in the JSDL. However, the file execute.bin doesn't exist in that location (and never does, at any point in time, I've checked with inotify). This contrasts with the inputs (Job0/inputs/0/0, etc) which do get written.
Any suggestions at this point?
Thanks. Best regards,
Jonas
Hey Akos,
Anything you can point out to help?
Cheers,
Jonas
Dear Jonas,
the URI looks a bit weird, but it shows the meaning. As you see it points to a servlet: "http://192.168.33.11:8080/storage/getFile" and one argument (path)is passed to it to get the file. I think this path is wrong because it does not point to the temp folder in where the real files staying.
So I would suggest to replace the path to the correct one.
And it can be automatized, so you can build this link creating the downloading process into your portlet.
Cheers,
akos