"
Team:Berkeley Software/BingNotebook/Notes
From 2009.igem.org
| (22 intermediate revisions not shown) | |||
| Line 1: | Line 1: | ||
| + | {{Template:BerkeleySoftwareHeader}} | ||
| + | |||
==Week One== | ==Week One== | ||
| Line 9: | Line 11: | ||
[[Image:DB_manager_notes1.png|center|frame|Proposed new Clotho infrastructure]] | [[Image:DB_manager_notes1.png|center|frame|Proposed new Clotho infrastructure]] | ||
| - | + | ==Week Two== | |
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 18:02, 9 June 2009 (UTC)=== | ||
| + | Yesterday we had our general meeting of the week, where we decided who would be doing their minipresentations when next week. I'll be going on Monday. Our last team member, Lesia, arrived in the afternoon and started getting acquainted with Clotho. | ||
| + | |||
| + | Nina and I also went to the wet lab again to talk to Jenn about more robot details. We figured out a few more things the robot automation piece will need to do: keep track of the amount of DNA needed to finish a set of assemblies and warn the user when there isn't enough sample left, and also we need to break the protocol used in Chris' lab down into pieces that are sufficiently general and configurable so they can be reused to easily create new robot automation protocols. | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 18:20, 12 June 2009 (UTC)=== | ||
| + | I had a meeting with Doug about the new data model on Wednesday, and he liked my idea of having a core set of keywords and then having a set of extension keyword files. Some features Doug want include automatic connectivity detection and alternate path usage. I should also come up with some toy examples we can refer to, instead of always drawing random diagrams that may or may not be relevant. | ||
| + | |||
| + | I should also be helping Nina get something working for the robot automation project this weekend for her deadline next Friday. | ||
| + | |||
| + | ==Week Three== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 06:27, 15 June 2009 (UTC)=== | ||
| + | I've done some work this weekend on filling out a data model that will be compatible (more or less) with the current JCA database and parts of the JBEI. I've also been researching existing persistence frameworks for java, and whether any of them are suitable to for use on this project. I'll post more after my minipresentation during the meeting tomorrow. | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 18:23, 19 June 2009 (UTC)=== | ||
| + | At the database meeting this week, I presented a complete high level picture of how the new Clotho will bring in data from the database. We will have a set of files that define the objects the rest of Clotho want to use, and what fields these objects should have. These files will get parsed and some internal structure of a data model will get built. The user will specify some database to connect to, and then a mapping module will use metadata from the database and user input to map database table names and column names to Clotho object and field keywords. The user will also specify the connectivity of the objects at this point. The mapped data model structure will be used by a code generator to build a java class for each object, along with a hibernate mapping file. These classes will all extend the Datum base class so the rest of Clotho can use them. The classes will be compiled dynamically, and then (hopefully) everything will work. | ||
| + | |||
| + | We also came up with an API for both the core and the Datum objects. | ||
| + | |||
| + | [[Image:BerkeleySoftware_clothoDBinfra1.png|center|frame|500px|Clotho Data]] | ||
| + | |||
| + | ==Week Four== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 17:37, 25 June 2009 (UTC)=== | ||
| + | This week I've been working on coding the java files that hibernate needs to work with the JCA database, since we want to be able to test the API. These classes will eventually be autogenerated and compiled by the piece that Evan is working on. I will be working on the mapping piece and the keyword parsing piece. I've decided to use XML for the keyword files, as there already exists XML parsing libraries, thus freeing myself from writing a parser for some syntax I come up with myself. I will also be finishing up the list of core keywords that Clotho core tools will use - this will be based mainly on the JBEI data model but also incorporate some fields that Chris wants. | ||
| + | |||
| + | Some other things that came up during this weeks database meeting: | ||
| + | * Every tool that uses the database should have some path selection piece | ||
| + | * Need to talk to Chris and Josh about how to represent composite parts in a database | ||
| + | * talk to Davidson asking for read access to their database | ||
| + | |||
| + | ==Week Five== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 00:14, 1 July 2009 (UTC)=== | ||
| + | At the beginning of this week, I had the data core working the the JCA database. The main goal for this week is to finish the mapping manager, and hope that we can start on the code generation piece next week. I've also just had a meeting with Chris and Josh about the core keyword list - their opinion is that as long as the list is easily extensible there shouldn't be any problems. We also talked about how to represent composite parts in the database - my opinion is that we should also store a flattened list of the basic parts as a field, so we're not required to store the entire construction tree of a part just to know its components. To compromise, both options will be in the core data keyword list. | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 00:59, 3 July 2009 (UTC)=== | ||
| + | Its now more than halfway through the week, and I'm almost halfway done...I should be farther along, but Monday and Tuesday were completely filled with meetings, and there were quite a few meetings yesterday too. | ||
| + | |||
| + | ==Week Six== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 23:17, 9 July 2009 (UTC)=== | ||
| + | This week will the last week of formal meetings - next week Doug will be helping produce code on various projects. The mapping manager is still not complete - its taking longer than I thought it would, due to various difficulties in obtaining metadata from the database. | ||
| + | |||
| + | ==Week Eight== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 00:07, 22 July 2009 (UTC)=== | ||
| + | This week I am continuing work on the path finding problem. I've found a java graph library, JGraphT, that I'll be using as a graph framework. I also found the java Preferences library, which I think we should use for Clotho preference data, instead of having a bunch of scattered files. | ||
| + | |||
| + | I emailed the Davidson database contact, Bill, and he's replied that he'll implement foreign keys in their database, and he'll send me an updated copy of their sql dump next week. | ||
| + | |||
| + | ==Week Nine== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 21:04, 31 July 2009 (UTC)=== | ||
| + | We've gotten a first pass at the database connection piece done - we can connect to a database, grab the columns, map them to the field keywords, and then dynamically generate the files needed for hibernate to function, and then use them to query the database. | ||
| + | |||
| + | This weekend I'll be setting up a sourceforge project that we're going to migrate clothoII into. The project will be at http://clothocad.sourceforge.net. I'm setting up the svn repository, moving the code we have, trying to organize it better, and editing the website. | ||
| + | |||
| + | I also attended the data exchange workshop at Stanford on the 26th with Doug. I got to listen to people from various organizations argue about what a part is, and in the end everyone agreed to at least attempt to get their tools to talk to other people's tools as a way to move forward instead of moving in circles trying to get people to agree. | ||
| + | |||
| + | ==Week Eleven== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 22:04, 7 August 2009 (UTC)=== | ||
| + | So we've more or less successfully moved into sourceforge. Most of the old tools have been migrated, except for the "Holy Trinity" of managers (parts manager, plate manager, and database manager). The core tool loading system has been revamped and is relatively stable. I still need to finish a tool import utility that will move your tool files into the correct places for you, and also generate the xml file required for tool loading. | ||
| + | |||
| + | We came up with a high level timeline of goals at our meeting yesterday, which is reproduced below: | ||
| + | |||
| + | <pre> | ||
| + | Aug | ||
| + | 1st week - tools migrated | ||
| + | 2nd - new tools, help/preferences, Evan leaves(finishes adjacent object paths) | ||
| + | 3rd - Icons | ||
| + | Sept - 3 managers, BioCAD migration | ||
| + | 1st - | ||
| + | 2nd - Chess demo | ||
| + | 3rd - Hilfinger/Paper, Davidson | ||
| + | 29th - Thien's 21st bday | ||
| + | Oct - India | ||
| + | 1st - Polish | ||
| + | 2nd - iGEM Bay Area/Presentation | ||
| + | 3rd - Poster/Wiki/clothocad.com/sourceforge | ||
| + | Doug 32nd bday | ||
| + | Jamboree!! | ||
| + | </pre> | ||
| + | |||
| + | For myself, I need to start polishing the UI for database connections, and also decide on a method for storing mappings. | ||
| + | |||
| + | ==Week Fourteen== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 23:49, 23 August 2009 (UTC)=== | ||
| + | Summer research for iGEM 2009 is winding down - we have our last monday meeting of the summer tomorrow morning, and then classes start on Wednesday. | ||
| + | |||
| + | I've finally "finished" the revamped database management system - you can navigate from one object to another using the API we developed at the beginning of the summer. This fall, I'll need to port over the old database manager from Clotho Classic (as we're now calling it), polish up the UI for the database management system, and make sure all my source code is cleaned up, organized, and well-commented. | ||
| + | |||
| + | ==Week Fifteen and Onwards== | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 17:24, 3 September 2009 (UTC)=== | ||
| + | Fall semester started last week, so I haven't really made much progress. With single hop object traversal working, I need to test it before Doug gets back from his trip to Michigan. I also plan to finish the saving a connection functionality before then, as this will make testing much easier for the whole team. | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 01:53, 10 September 2009 (UTC)=== | ||
| + | Today Doug returns from his vacation, and we had our meeting. It seems that the wet lab is not very happy with Clotho as it is now, since from their perspective Clotho has made backwards progress since the release of Clotho Classic Beta. We haven't moved the most important parts over - the parts manager and database manager, and the database connection/binding process is not polished yet. The database manager and binding/connection process are what I need to finish by next week. | ||
| + | |||
| + | ===[[User:Sixpi|Sixpi]] 05:57, 20 October 2009 (UTC)=== | ||
| + | 2 weeks until the jamboree! And only 2 days until the wiki deadline! | ||
| + | |||
| + | We presented at the NorCal iGEM Jamboree last Saturday, and it went pretty well, considering we completely reorganized our presentation the night before =D. I ended up doing the intro slides and then the data model first, so Friday night/early Saturday morning was trying to practice my part.... >.< | ||
| + | |||
| + | Anyways, we should have no problem finishing the wiki by Wednesday. After that's done, we just have to finish the poster (Doug claims this is not hard), and also clean up the code/make sure everything is working/fix up the sourceforge page. | ||
---- | ---- | ||
[[Team:Berkeley_Software/Notebook|Back to Notebooks]] | [[Team:Berkeley_Software/Notebook|Back to Notebooks]] | ||
Latest revision as of 05:57, 20 October 2009
Contents |
Week One
Sixpi 18:34, 2 June 2009 (UTC)
iGEM starts again for me! Last year I was on the Berkeley wetlab team - this year, I'll be working on the software tools team. This week, we're getting a bunch of tutorials from Doug to get everyone caught up with where Clotho is now, and also thinking about which of the subproject(s) we want to talk about. I will probably be working on database support, and I would like to help with the robot automation/assembly algorithm project. I've put up a short bio of myself here.
Sixpi 15:55, 3 June 2009 (UTC)
I talked with Doug yesterday about how the DB manager should work for the new Clotho. The new DB manager won't be a rewrite of what I have now - what I have now should probably be renamed the DB browser. The new DB manager will be the default gateway for information to travel in and out of Clotho, so we want it to be as flexible and powerful, and yet as easy to use as possible. Out of our discussion, we decided that a binding file of some sort was still needed, and that we needed some list of internal types/keywords that all Clotho tools should understand, and a binding file to translate between external types and internal types. Once data passes through the binding manager, another piece of code will be responsible for checking the external data model to see if it is compatible enough with the Clotho data model to let the tools do their work. We will need to consider some things about how different the topologies of the external and internal data models are, and how different the models can be before the tools will not be able to traverse the external data intelligently.
Week Two
Sixpi 18:02, 9 June 2009 (UTC)
Yesterday we had our general meeting of the week, where we decided who would be doing their minipresentations when next week. I'll be going on Monday. Our last team member, Lesia, arrived in the afternoon and started getting acquainted with Clotho.
Nina and I also went to the wet lab again to talk to Jenn about more robot details. We figured out a few more things the robot automation piece will need to do: keep track of the amount of DNA needed to finish a set of assemblies and warn the user when there isn't enough sample left, and also we need to break the protocol used in Chris' lab down into pieces that are sufficiently general and configurable so they can be reused to easily create new robot automation protocols.
Sixpi 18:20, 12 June 2009 (UTC)
I had a meeting with Doug about the new data model on Wednesday, and he liked my idea of having a core set of keywords and then having a set of extension keyword files. Some features Doug want include automatic connectivity detection and alternate path usage. I should also come up with some toy examples we can refer to, instead of always drawing random diagrams that may or may not be relevant.
I should also be helping Nina get something working for the robot automation project this weekend for her deadline next Friday.
Week Three
Sixpi 06:27, 15 June 2009 (UTC)
I've done some work this weekend on filling out a data model that will be compatible (more or less) with the current JCA database and parts of the JBEI. I've also been researching existing persistence frameworks for java, and whether any of them are suitable to for use on this project. I'll post more after my minipresentation during the meeting tomorrow.
Sixpi 18:23, 19 June 2009 (UTC)
At the database meeting this week, I presented a complete high level picture of how the new Clotho will bring in data from the database. We will have a set of files that define the objects the rest of Clotho want to use, and what fields these objects should have. These files will get parsed and some internal structure of a data model will get built. The user will specify some database to connect to, and then a mapping module will use metadata from the database and user input to map database table names and column names to Clotho object and field keywords. The user will also specify the connectivity of the objects at this point. The mapped data model structure will be used by a code generator to build a java class for each object, along with a hibernate mapping file. These classes will all extend the Datum base class so the rest of Clotho can use them. The classes will be compiled dynamically, and then (hopefully) everything will work.
We also came up with an API for both the core and the Datum objects.
Week Four
Sixpi 17:37, 25 June 2009 (UTC)
This week I've been working on coding the java files that hibernate needs to work with the JCA database, since we want to be able to test the API. These classes will eventually be autogenerated and compiled by the piece that Evan is working on. I will be working on the mapping piece and the keyword parsing piece. I've decided to use XML for the keyword files, as there already exists XML parsing libraries, thus freeing myself from writing a parser for some syntax I come up with myself. I will also be finishing up the list of core keywords that Clotho core tools will use - this will be based mainly on the JBEI data model but also incorporate some fields that Chris wants.
Some other things that came up during this weeks database meeting:
- Every tool that uses the database should have some path selection piece
- Need to talk to Chris and Josh about how to represent composite parts in a database
- talk to Davidson asking for read access to their database
Week Five
Sixpi 00:14, 1 July 2009 (UTC)
At the beginning of this week, I had the data core working the the JCA database. The main goal for this week is to finish the mapping manager, and hope that we can start on the code generation piece next week. I've also just had a meeting with Chris and Josh about the core keyword list - their opinion is that as long as the list is easily extensible there shouldn't be any problems. We also talked about how to represent composite parts in the database - my opinion is that we should also store a flattened list of the basic parts as a field, so we're not required to store the entire construction tree of a part just to know its components. To compromise, both options will be in the core data keyword list.
Sixpi 00:59, 3 July 2009 (UTC)
Its now more than halfway through the week, and I'm almost halfway done...I should be farther along, but Monday and Tuesday were completely filled with meetings, and there were quite a few meetings yesterday too.
Week Six
Sixpi 23:17, 9 July 2009 (UTC)
This week will the last week of formal meetings - next week Doug will be helping produce code on various projects. The mapping manager is still not complete - its taking longer than I thought it would, due to various difficulties in obtaining metadata from the database.
Week Eight
Sixpi 00:07, 22 July 2009 (UTC)
This week I am continuing work on the path finding problem. I've found a java graph library, JGraphT, that I'll be using as a graph framework. I also found the java Preferences library, which I think we should use for Clotho preference data, instead of having a bunch of scattered files.
I emailed the Davidson database contact, Bill, and he's replied that he'll implement foreign keys in their database, and he'll send me an updated copy of their sql dump next week.
Week Nine
Sixpi 21:04, 31 July 2009 (UTC)
We've gotten a first pass at the database connection piece done - we can connect to a database, grab the columns, map them to the field keywords, and then dynamically generate the files needed for hibernate to function, and then use them to query the database.
This weekend I'll be setting up a sourceforge project that we're going to migrate clothoII into. The project will be at http://clothocad.sourceforge.net. I'm setting up the svn repository, moving the code we have, trying to organize it better, and editing the website.
I also attended the data exchange workshop at Stanford on the 26th with Doug. I got to listen to people from various organizations argue about what a part is, and in the end everyone agreed to at least attempt to get their tools to talk to other people's tools as a way to move forward instead of moving in circles trying to get people to agree.
Week Eleven
Sixpi 22:04, 7 August 2009 (UTC)
So we've more or less successfully moved into sourceforge. Most of the old tools have been migrated, except for the "Holy Trinity" of managers (parts manager, plate manager, and database manager). The core tool loading system has been revamped and is relatively stable. I still need to finish a tool import utility that will move your tool files into the correct places for you, and also generate the xml file required for tool loading.
We came up with a high level timeline of goals at our meeting yesterday, which is reproduced below:
Aug 1st week - tools migrated 2nd - new tools, help/preferences, Evan leaves(finishes adjacent object paths) 3rd - Icons Sept - 3 managers, BioCAD migration 1st - 2nd - Chess demo 3rd - Hilfinger/Paper, Davidson 29th - Thien's 21st bday Oct - India 1st - Polish 2nd - iGEM Bay Area/Presentation 3rd - Poster/Wiki/clothocad.com/sourceforge Doug 32nd bday Jamboree!!
For myself, I need to start polishing the UI for database connections, and also decide on a method for storing mappings.
Week Fourteen
Sixpi 23:49, 23 August 2009 (UTC)
Summer research for iGEM 2009 is winding down - we have our last monday meeting of the summer tomorrow morning, and then classes start on Wednesday.
I've finally "finished" the revamped database management system - you can navigate from one object to another using the API we developed at the beginning of the summer. This fall, I'll need to port over the old database manager from Clotho Classic (as we're now calling it), polish up the UI for the database management system, and make sure all my source code is cleaned up, organized, and well-commented.
Week Fifteen and Onwards
Sixpi 17:24, 3 September 2009 (UTC)
Fall semester started last week, so I haven't really made much progress. With single hop object traversal working, I need to test it before Doug gets back from his trip to Michigan. I also plan to finish the saving a connection functionality before then, as this will make testing much easier for the whole team.
Sixpi 01:53, 10 September 2009 (UTC)
Today Doug returns from his vacation, and we had our meeting. It seems that the wet lab is not very happy with Clotho as it is now, since from their perspective Clotho has made backwards progress since the release of Clotho Classic Beta. We haven't moved the most important parts over - the parts manager and database manager, and the database connection/binding process is not polished yet. The database manager and binding/connection process are what I need to finish by next week.
Sixpi 05:57, 20 October 2009 (UTC)
2 weeks until the jamboree! And only 2 days until the wiki deadline!
We presented at the NorCal iGEM Jamboree last Saturday, and it went pretty well, considering we completely reorganized our presentation the night before =D. I ended up doing the intro slides and then the data model first, so Friday night/early Saturday morning was trying to practice my part.... >.<
Anyways, we should have no problem finishing the wiki by Wednesday. After that's done, we just have to finish the poster (Doug claims this is not hard), and also clean up the code/make sure everything is working/fix up the sourceforge page.
