Team:Berkeley Software/KeplerTutorial

From 2009.igem.org

(Difference between revisions)
 
(42 intermediate revisions not shown)
Line 1: Line 1:
{{Template:BerkeleySoftwareHeader}}
{{Template:BerkeleySoftwareHeader}}
{{Template:BerkeleySoftwareProject
{{Template:BerkeleySoftwareProject
 +
|video=
 +
<center><html><object width="560" height="340"><param name="movie" value="http://www.youtube.com/v/E3bEai9LClM&hl=en&fs=1&rel=0"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/E3bEai9LClM&hl=en&fs=1&rel=0" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="560" height="340"></embed></object></html></center>
 +
|rightSideBox=
|rightSideBox=
Line 7: Line 10:
|title=
|title=
Kepler Tutorial
Kepler Tutorial
 +
 +
__NOTOC__
|intro=
|intro=
-
Kepler helps scientists design models and analyses across a broad range of scientific and engineering disciplines.[https://kepler-project.org/ Kepler Project Website] This summer we introduced Kepler into synthetic biology by building a set of new Kepler actors for assembly automation and Clotho connection.
+
The Kepler design environment helps scientists design models and perform analysis across a broad range of scientific and engineering disciplines. We were interested in answering the question, ''"can Kepler be used in synthetic biology efficiently"''. In order to investigate this more we introduced Kepler into synthetic biology by building a set of new Kepler actors for assembly automation and to provide a remote connection to Clotho. Our goal was to create a set of reusable actors and workflows which in the face of constant change could adapt to reflect the latest lab protocols and design flows. For more information on Kepler we invite you to check out the [http://kepler-project.org/ Kepler Project Website]. In addition, much of the work we performed was with the [http://chess.eecs.berkeley.edu Center for Hybrid Embedded Software Systems (CHESS)] as Kepler was born of the [http://ptolemy.eecs.berkeley.edu/ Ptolemy Project].
|content=
|content=
-
 
==Components==
==Components==
In Kepler, a workflow is built from a collection of process steps that run under the control of a supervisor system.<br>
In Kepler, a workflow is built from a collection of process steps that run under the control of a supervisor system.<br>
-
Those separate steps are called "actors", they are represented as square icons but inside them are codes to define what to run in a step. A step can be an input/output operation, a computational function, or even another workflow. These actors are supervised by the "director", which keep track of when to run each actor.<br>
 
-
[[Image:BerkeleySoftware_KeplerActorList.png|center|Some actors]]
 
-
Some Kepler actors
 
-
To start designing Kepler workflow, the online manuals provide details instruction: [https://kepler-project.org/users/documentation Kepler Documentation]<br>
+
Those separate steps are called "actors", they are represented as square icons but inside them resides code determining what to run in a "step" of a workflow simulation. A step can be an input/output operation, a computational function, or even another workflow. Actors can support hierarchical construction where composite actors contain multiple primitive actors. These actors are supervised by the "director", which keeps track of when to run each actor.<br>
 +
 
 +
[[Image:BerkeleySoftware_KeplerActorList.png|thumb|left|530px|<span style="font-family: Georgia, serif;">Some Sample Kepler Actors</span>]]
 +
[[Image:BerkeleySoftware_KeplerDirectors.png|thumb|left|170px|<span style="font-family: Georgia, serif;">Common Kepler Directors - Static DataFlow and Dynamic DataFlow</span>]]
 +
 
 +
<br clear="all">
 +
 
 +
Kepler comes with a library of available actors for many tasks, and user can always write their custom actors and share.
 +
 
 +
The online manuals provide detailed instruction for installing and making workflows: [https://kepler-project.org/users/documentation Kepler Documentation]. We highly recommend those interested read this material as it contains much more detailed descriptions then we will provide here.<br>
 +
 
 +
==Automated Assembly Workflow==
 +
<center>[[Image:BerkeleySoftware_AutomationNoted.png|thumb|left|900px|<span style="font-family: Georgia, serif;">Our Automated Assembly Kepler Workflow</span>]]</center>
 +
<br clear="all">
 +
As described in the [https://2009.igem.org/Team:Berkeley_Software/Kepler Kepler project page], this workflow receives  assembly information from Clotho's Algorithm Manager, and outputs files for the liquid handling robot as well as human readable instructions.<br>
 +
 
 +
===Detailed Actor List===
 +
<b>A</b>. OpenClothoConnection: connect Kepler to Clotho while the Clotho RMI Tool is running. If successful, this actor will output the connection with Clotho RMI methods so the next actors can use them.<br>
 +
<b>B</b>. GetAssemblyGraph & GetString: both receive data from Clotho, such as an assembly graph or some debugging information. These actors output either genetic Object data or a special structure such as the assembly information.<br>
 +
<b>C</b>. GraphProcessing: process the assembly information, check & prepare data for further options.<br>
 +
<b>D</b>. DialogOption: allows for users to make antibiotic choices, either at runtime or the user can provide a choice as a parameter before workflow runs.<br>
 +
<b>E</b>. AutomationScheduler<br>
 +
<b>F</b>. Constant: Input for the stage number. This is only a constant number, thus it is easy for future modifications such as using an array so that multiple stages can be specified in one run.<br>
 +
<b>G</b>. OneStageProcessing: Creates the files from an assembly graph, the choice of stage, and antibiotic selection.<br>
 +
<b>H</b>. Display actors: Use to display information, such as when the workflow is finished getting data and the names of created files.<br>
 +
 
 +
 
 +
==Clotho RMI Connection==
 +
One of the key challenges for biology tool development is how to effectively share and cooperate. This year we strengthened the possibility of establishing a connection between Clotho and other software tools. In particular, the Clotho platform and Kepler can connect and send data through the a remote API interface via Java RMI. This RMI connection will guarantee Clotho and Kepler to talk to any Java software that import the same interface.
 +
 
 +
<center>[[Image:BerkeleySoftware_ClothoConnection.png|left|thumb|550px|<span style="font-family: Georgia, serif;">RMI Connection and Clotho Interaction</span>]]<br></center>
 +
<br clear="all">
 +
 
 +
In our assembly workflow, we made a custom actor which opens a Clotho RMI connection. We also made actors that read data from Clotho tools. These actors support communication via genetic data objects (for maximum flexibility) and have parameters for tool name and object fields, thus they will work with any future Clotho tools, as long as those tools support [https://2009.igem.org/Team:Berkeley_Software/Data_Model Clotho Data API] methods.
 +
 
 +
Clotho RMI interface:<br>
 +
<code>
 +
    /**
 +
    * Get data from a Clotho tool. The tool must already instantiated in Clotho.
 +
    * Data will be called from the core method getData
 +
    * @param toolName - exact class name in up & lower case of a Clotho tool
 +
    * @param object
 +
    * @param field
 +
    * @return data as Object, need to cast back to the correct class
 +
    * @throws java.rmi.RemoteException
 +
    */
 +
    public Object getData(String toolName, String object, String field) throws RemoteException;
 +
    /**
 +
    * Send data to an instantiated Clotho tool.
 +
    * Data will be sent by the core method sendData without operation code
 +
    * @param toolName -exact class name in up & lower case of a  Clotho tool
 +
    * @param object
 +
    * @param field
 +
    * @param data - Data to be sent
 +
    * @throws java.rmi.RemoteException
 +
    */
 +
    public void sendData(String toolName, String object, String field, Object data) throws RemoteException;
 +
 
 +
</code>
 +
<center>[[Image:BerkeleySoftware_KeplerGetString.png|thumb|left|700px|<span style="font-family: Georgia, serif;">"GetString" Actor Interface</span>]]</center><br>
 +
<br clear="all">
 +
This is the configuration of the Kepler actor "GetString". This actor can read data from tools in Clotho and send out a String object to output ports. In the image, we want to read data from the tool "AlgorithmManager", if there are multiple data fields, we can further specify what data we want with the other two parameters.
 +
 
 +
==User Options==
 +
In an assembly workflow, we need to to have some user options such as antibiotic selection or stage choices. The problem during a running workflow is to allow the user the ability to select a choice at runtime, along with displaying some information about the current data so the user knows as much as possible about about their preferred choices. However, if the same workflow needs to run multiple times, with almost the same data and choices, repeatedly asking the user to click similar things repeatedly is not ideal. With Kepler workflows, we present two ways of giving users choices, either at runtime or via parameters before workflow starts. For example, the "Dialog Option" actor for antibiotic choices will pop up a dialog for choosing antibiotics at runtime, after Kepler receives the assembly graph and provides an explanation about each choice. Alternatively, experienced users can also input an integer as an antibiotic index before clicking run, and the workflow will run to completion without pause for a pop-up dialog box. These dual ways of presenting user options enable the workflow to run as a regular tool with step-by-step choices and a fully automated process for a large numbers of similar runs, even without graphical interface.
 +
 
 +
<center>[[Image:BerkeleySoftware_AntibioticChoices.png|thumb|left|600px|<span style="font-family: Georgia, serif;">"Dialog Option" Actor Illustration for Antibiotic Selection</span>]]</center><br>
 +
<br clear="all">
-
==Clotho Connection==
+
The stage chooser also is designed with flexibility in mind, allowing users to further modify the stage choice by putting an array of stage numbers, or another dialog option.
-
One of the key challenge for biology tool development is how to effectively share and cooperate. This year we strengthen the possibility of connection between multiple software. In particular, the Clotho platform from last year
+
-
and Kepler can connect and send data through the remore interface with Java RMI. This RMI connection will guarantee Clotho and Kepler to talk to any Java software that import the same interface.
+
-
[[Image:BerkeleySoftwareKepler_GetString: GetString actor]]
+
-
This is the configuration of the Kepler actor GetString. This actor can read data as String from any tools in Clotho that provide a public data access method. In the image, we want to read data from the tool AlgorithmManager, if there are multiple data fields, we can further specify what data we want with the other parameters.
+
-
(List of RMI Clotho method)<br>
+
==Demo==
-
(screenshot of Clotho - Kepler with arrow)<br>
+
<html>
 +
<center>
 +
<caption> Demo illustrating a Kepler workflow in action.</caption>
 +
<embed
 +
  src="https://static.igem.org/mediawiki/2009/8/87/BerkeleySoftware_KeplerWink.swf"
 +
  width="950"
 +
  height="800"
 +
  allowscriptaccess="always"
 +
  allowfullscreen="true"
 +
  autoplay="false"
 +
/>
 +
</center>
 +
</html>
}}
}}

Latest revision as of 00:32, 22 October 2009


Kepler Tutorial

Introduction

The Kepler design environment helps scientists design models and perform analysis across a broad range of scientific and engineering disciplines. We were interested in answering the question, "can Kepler be used in synthetic biology efficiently". In order to investigate this more we introduced Kepler into synthetic biology by building a set of new Kepler actors for assembly automation and to provide a remote connection to Clotho. Our goal was to create a set of reusable actors and workflows which in the face of constant change could adapt to reflect the latest lab protocols and design flows. For more information on Kepler we invite you to check out the [http://kepler-project.org/ Kepler Project Website]. In addition, much of the work we performed was with the [http://chess.eecs.berkeley.edu Center for Hybrid Embedded Software Systems (CHESS)] as Kepler was born of the [http://ptolemy.eecs.berkeley.edu/ Ptolemy Project].



Components

In Kepler, a workflow is built from a collection of process steps that run under the control of a supervisor system.

Those separate steps are called "actors", they are represented as square icons but inside them resides code determining what to run in a "step" of a workflow simulation. A step can be an input/output operation, a computational function, or even another workflow. Actors can support hierarchical construction where composite actors contain multiple primitive actors. These actors are supervised by the "director", which keeps track of when to run each actor.

Some Sample Kepler Actors
Common Kepler Directors - Static DataFlow and Dynamic DataFlow


Kepler comes with a library of available actors for many tasks, and user can always write their custom actors and share.

The online manuals provide detailed instruction for installing and making workflows: Kepler Documentation. We highly recommend those interested read this material as it contains much more detailed descriptions then we will provide here.

Automated Assembly Workflow

Our Automated Assembly Kepler Workflow


As described in the Kepler project page, this workflow receives assembly information from Clotho's Algorithm Manager, and outputs files for the liquid handling robot as well as human readable instructions.

Detailed Actor List

A. OpenClothoConnection: connect Kepler to Clotho while the Clotho RMI Tool is running. If successful, this actor will output the connection with Clotho RMI methods so the next actors can use them.
B. GetAssemblyGraph & GetString: both receive data from Clotho, such as an assembly graph or some debugging information. These actors output either genetic Object data or a special structure such as the assembly information.
C. GraphProcessing: process the assembly information, check & prepare data for further options.
D. DialogOption: allows for users to make antibiotic choices, either at runtime or the user can provide a choice as a parameter before workflow runs.
E. AutomationScheduler
F. Constant: Input for the stage number. This is only a constant number, thus it is easy for future modifications such as using an array so that multiple stages can be specified in one run.
G. OneStageProcessing: Creates the files from an assembly graph, the choice of stage, and antibiotic selection.
H. Display actors: Use to display information, such as when the workflow is finished getting data and the names of created files.


Clotho RMI Connection

One of the key challenges for biology tool development is how to effectively share and cooperate. This year we strengthened the possibility of establishing a connection between Clotho and other software tools. In particular, the Clotho platform and Kepler can connect and send data through the a remote API interface via Java RMI. This RMI connection will guarantee Clotho and Kepler to talk to any Java software that import the same interface.

RMI Connection and Clotho Interaction


In our assembly workflow, we made a custom actor which opens a Clotho RMI connection. We also made actors that read data from Clotho tools. These actors support communication via genetic data objects (for maximum flexibility) and have parameters for tool name and object fields, thus they will work with any future Clotho tools, as long as those tools support Clotho Data API methods.

Clotho RMI interface:

   /**
    * Get data from a Clotho tool. The tool must already instantiated in Clotho.
    * Data will be called from the core method getData
    * @param toolName - exact class name in up & lower case of a Clotho tool
    * @param object
    * @param field
    * @return data as Object, need to cast back to the correct class
    * @throws java.rmi.RemoteException
    */
   public Object getData(String toolName, String object, String field) throws RemoteException;
   /**
    * Send data to an instantiated Clotho tool.
    * Data will be sent by the core method sendData without operation code
    * @param toolName -exact class name in up & lower case of a  Clotho tool
    * @param object
    * @param field
    * @param data - Data to be sent
    * @throws java.rmi.RemoteException
    */
   public void sendData(String toolName, String object, String field, Object data) throws RemoteException;

"GetString" Actor Interface


This is the configuration of the Kepler actor "GetString". This actor can read data from tools in Clotho and send out a String object to output ports. In the image, we want to read data from the tool "AlgorithmManager", if there are multiple data fields, we can further specify what data we want with the other two parameters.

User Options

In an assembly workflow, we need to to have some user options such as antibiotic selection or stage choices. The problem during a running workflow is to allow the user the ability to select a choice at runtime, along with displaying some information about the current data so the user knows as much as possible about about their preferred choices. However, if the same workflow needs to run multiple times, with almost the same data and choices, repeatedly asking the user to click similar things repeatedly is not ideal. With Kepler workflows, we present two ways of giving users choices, either at runtime or via parameters before workflow starts. For example, the "Dialog Option" actor for antibiotic choices will pop up a dialog for choosing antibiotics at runtime, after Kepler receives the assembly graph and provides an explanation about each choice. Alternatively, experienced users can also input an integer as an antibiotic index before clicking run, and the workflow will run to completion without pause for a pop-up dialog box. These dual ways of presenting user options enable the workflow to run as a regular tool with step-by-step choices and a fully automated process for a large numbers of similar runs, even without graphical interface.

"Dialog Option" Actor Illustration for Antibiotic Selection


The stage chooser also is designed with flexibility in mind, allowing users to further modify the stage choice by putting an array of stage numbers, or another dialog option.

Demo

Demo illustrating a Kepler workflow in action.