Welcome to twSnowfall project!

Extended functionality and usability for distributed code that is based on the snowfall package.

Download/Install

Documentaion

The package comes with documentaion and examples. Within R type: 
library(twSnowfall)
?twSnowfall

Extended load balancing

In the following example there are two arguments of the remote function that receive different values per node. 
#sfInit(parallel=TRUE,cpus=2)

X <- matrix(0:9,nrow=5,ncol=2)
dimnames(X) <- list( rows=paste("r",1:nrow(X),sep=""), cols=paste("c",1:ncol(X),sep="")  )
Y <- X*10
# delivering row i of X and row i of Y as arguments to F_Apply
F_ARGS <- function(i){list(arg1=X[i,],arg2=Y[i,])}
F_APPLY <- paste
.res <- sfFArgsApplyLB( nrow(X), F_ARGS, F_APPLY, sep="-")

Allowing dependencies of arguments to remote function on former results.

In the following example, the evaluation of a remote function for entries in one column of a matrix depends on the result of the former column. 
#sfInit(parallel=TRUE,cpus=2)

# using as many cpus as rows in Z
(Z<-matrix(letters[1:12],nrow=3))
F_APPLY <- function(x,z) paste(x,z,sep="");
F_ARGS <- function(i,prevRes){list(x=prevRes,z=Z[i])}	
.res0 <- rep("_",nrow(Z))	# dependStep will be length of .res0
resSeq <- sfFArgsApplyDep( length(Z), F_ARGS, F_APPLY, .res0)
(res <- matrix(sfSimplifyLBResult(resSeq),nrow=nrow(Z)))
This gives the same results as having one parallel call per column. However, in this implementation, the finished nodes do not need to wait for possibly slower finishing of other rows, and can tackle already further columns.

Simplifying debugging of remote errors.

In the following example we use a wrapper function in the remote process that catches errors and creates a dump file. This dump file then can be inspected in an interactive R-session for debugging. 
#sfInit(parallel=TRUE,cpus=2)
sfExport("sfRemoteWrapper",namespace="twSnowfall")

suppressWarnings(dir.create("tmp"))	# will store to tmp subdirectory
.remoteDumpfileBasename=file.path("tmp","testDump2")
.remoteDumpfile <- paste(.remoteDumpfileBasename,".rda",sep="")
unlink(.remoteDumpfile)
# throwing an error on remote process 
fTestStop <- function(){ stop("test throwing an error") }
tmp <- try( sfClusterCall( sfRemoteWrapper, remoteFun=fTestStop, remoteDumpfileBasename=.remoteDumpfileBasename ) )
# inspecting what was wrong in interactive R-session
load(.remoteDumpfile)
#debugger(get(.remoteDumpfileBasename))

miscellaneous functions

Several functions of general usability have been factored out into a separate package twMisc for usage by other packages.

The project summary page you can find here.