SECTOR / News: Recent posts

a step closer to the production quality.

Sector distributed file system: sector.sf.net

Sector: A distributed file system and parallel data processing framework (http://sector.sf.net)

We have released Sector version 2.5. This is a reliable version that includes many improvements and bug fixes after several month's work from the Sector team members. We also would like to thank our users and customers, who have provided many valuable feedbacks and suggestions.

We have a new member gnailzenh to help develop the Sector system. More contributors are welcome.

Sector: A Distributed File System and Parallel Data Processing Engine
http://sector.sf.net

Version 2.1 is a bug fix version for the recent 2.0 release. We fixed several critical bugs including: 1) replica was not created; 2) lost slave couldn't be restarted; 3) occasional crash with multiple masters.

Version 1.24 includes support for multiple active-active master servers and Sphere Streaming. Like Hadoop Streaming, Sphere Streaming allows users to run applications (including system commands) directly on Setcor files. The applications can be written in any programming languages. In addition, we have changed to BSD license to give users maximum flexibility in using Sector/Sphere.

Version 1.23 fixed a serious bug in 1.22. If you tried 1.22 and found deadlock problems with Sphere, please use 1.23. Thanks!

Sector: distributed data storage and processing
sector.sf.net

Sector: distributed data storage and processing system (http://sector.sf.net)

Version 1.22 includes several improvements on reliability and performance. In particular, the new version also refined the FUSE support and added local IO optimization.

Sector: distributed data storage and processing system (http://sector.sf.net)

Version 1.21 supports persistent data channel between slaves and clients, thus reduce the data connection set up time.

We also added FUSE file system interface to Sector, so that you can mount Sector as a local file system directory.

Version 1.21 also includes many bug fixes since version 1.20.

Sector: distributed data storage and processing system
http://sector.sourceforge.net/

Version 1.19 improves the client file access tools, the Sphere client scheduler, and many bug fixes. The documentation is also improved.

SECTOR: A Distributed Storage and Computing Infrastructure

Sector version 1.18 contains several improvements on the file system, in particular the real time replication of data update.

Project website: http://sector.sourceforge.net

Sector is a distributed storage and data processing platform. Version 1.17 added more detailed documentations and various bug fixes. In addition, Sector now supports secure data transfer in this version.

Sector supports distributed storage and data processing, similar to GFS/MapReduce and Hadoop. Unlike Hadoop, Sector is written in C++. Sector also supports wide area deployment. The MapReduce counterpart in Sector is called Sphere, which supports user defined functions on data segments. The user defined functions can be either Map, Reduce, or else.

We have demonstrated Sector/Sphere in the Supercomputing conference last month in Austin, TX, and won the bandwidth challenge award.

Nov. 20, 2008

SuperComputing 2008

Austin, TX

The Bandwidth Challenge: Unleashing the Power of the Network

The National Center for Data Mining (NCDM) at UIC and the Open Cloud Consortium were awarded the 2008 SC08 Bandwidth Challenge award at SC 08 today in Austin.

Their entry was titled "Towards Global Scale Cloud Computing: Using Sector and Sphere on the Open Cloud Testbed" and was led by Dr. Yunhong Gu of the University of Illinois at Chicago and Dr. Robert Grossman of the University of Illinois at Chicago and Open Data Group.... read more

MapReduce is supported now.

many bug fixes. we are doing well towards v2.0.

more reliable. bug fixes. more tools.

version 1.6 introduces security and centralized control.

A lot of improvement and bug fix from version 1.4, especially on Sphere. SQL access is removed due lack of human resource.

Including bug fixes, especially on the Sphere functions.

If you are trying to download SDSS data, please go to sdss.ncdm.uic.edu/download.html for instructions. The Sector client software for SDSS can be downloaded via that page.

Please do NOT use the Sector releases from SourceForge for the SDSS download. It may not work with the special SDSS Sector server.

SECTOR: A Distributed Storage and Computing Infrastructure

Version 1.3 is a major step torwards the first reliable production version. It contains many bug fixed, code clean up, and complete Sphere client support. We plan to release version 2.0 before SC07.

this is the first stable version. We will add more features soon...

We are on the way!

The first alpha version is released. No documentation yet. Please write to me directly if you have any questions.

the first prototype is in SVN now. still a very native version, but more to come soon!