scalaFX - Titledpane: how do I get the heigth of the content? - height

I created a TiteldPane in scalafx
val titled: TitledPane = new TitledPane()
and put some nodes in it for my GUI.
Later I want to read out the heigth of the content of titled.
In javaFX this would be done with:
((Region) titled.getContent()).getHeight()
But if I try to read the height of the content in scala with:
titled.content.height
the height is marked as deprecated and does not compile. I've got a hint to github (scalafx/issue69) that explains why it is deprecated but does not explain how it can be done instead.
Just to clarify: I want to read out the height of the content of the titledpane, not just titled.heigth.
When titled is closed, then titled.height is 0, but I want to know what it would be when it is expanded (to detect when it has finished expanding actually).
So, how can I do this in scalafx?
EDIT:
Here is a example that shows the described error
import scalafx.Includes._
import scalafx.application.JFXApp
import scalafx.beans.property.DoubleProperty
import scalafx.beans.value.ObservableValue
import scalafx.collections.ObservableBuffer
import scalafx.event.ActionEvent
import scalafx.scene.Scene
import scalafx.scene.control.cell.TextFieldListCell
import scalafx.scene.control.{Button, ListView, TitledPane}
import scalafx.scene.layout.BorderPane
object TitledPaneEndOfExpansion extends JFXApp {
val expandedHeight = new DoubleProperty()
val data: ObservableBuffer[String] = new ObservableBuffer[String]() ++= List("some", "content", "for", "testing")
stage = new JFXApp.PrimaryStage {
title = "JavaFX: edit after rendering test"
val list: ListView[String] = new ListView[String](data) {
editable = true
cellFactory = TextFieldListCell.forListView()
height.onChange { (source: ObservableValue[Double, Number], oldValue: Number, newValue: Number) =>
expandedHeight.value = titled.content.height
println("old height is: " + oldValue.doubleValue() + " new height is: " + newValue.doubleValue())
if (newValue.doubleValue() == expandedHeight.value) {
edit(1)
}
}
}
val titled: TitledPane = new TitledPane {
text = "titled"
content = list
}
scene = new Scene {
root = new BorderPane {
center = titled
bottom = new Button() {
text = "edit cell 1"
onAction = { _: ActionEvent => list.edit(1) }
}
}
}
expandedHeight.value = titled.content.height //set to 400
list.edit(1)
}
}
And here is the buid.sbt file:
name := "JavaFXrenderingProblem"
version := "0.1"
scalaVersion := "2.13.3"
libraryDependencies += "org.scalafx" %% "scalafx" % "15.0.1-R21"
libraryDependencies += "org.controlsfx" % "controlsfx" % "8.40.18"
// Prevent startup bug in JavaFX
fork := true
// Tell Javac and scalac to build for jvm 1.8
javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
scalacOptions += "-target:jvm-1.8"
scalacOptions += "-feature"
When I just compile with plain sbt i get the compile error-message:
[info] compiling 1 Scala source to ... JavaFXrenderingProblem\target\scala-2.13\classes ...
[error] ... JavaFXrenderingProblem\src\main\scala\TitledPaneEndOfExpansion.scala:38:47: value height is not a member of scalafx.beans.property.ObjectProperty[javafx.scene.Node]
[error] expandedHeight.value = titled.content.height
[error] ^
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 3 s, completed 03.05.2021 11:09:02

I actually get two errors when I execute sbt run on your code, and I do not get a deprecation error:
[info] compiling 1 Scala source to C:\Users\SomeUser\src\SFC\target\scala-2.13\classes ...
[error] C:\Users\SomeUser\src\SFX\src\main\scala\TitledPaneEndOfExpansion.scala:23:41: value height is not a member of scalafx.beans.property.ObjectProperty[javafx.scene.Node]
[error] expandedHeight.value = titled.content.height
[error] ^
[error] C:\Users\MichaelAllen\src\SOSFX\src\main\scala\TitledPaneEndOfExpansion.scala:45:40: value height is not a member of scalafx.beans.property.ObjectProperty[javafx.scene.Node]
[error] expandedHeight.value = titled.content.height //set to 400
[error] ^
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 3 s, completed May 3, 2021 9:58:00 AM
From your code, the list value returns the contents of the TitledPane instance, titled, as a ListView[String]. It is this object whose height method you're trying to call. Correct?
The primary problem is that the content method of titled doesn't know enough about the type of the object that titled is storing. All it knows is that it is derived from javafx.scene.Node. Such Node instances do not have a height property, and hence your errors. (It's actually a little more complicated than that, but that's the simplest way to explain the issue.)
However, you already have a reference to the object that is the content of titled: list. So you can replace the second reference to titled.content.height with list.height. The first reference, in list's height's onChanged method, is accessible through the source parameter (it identifies the property that changed value, namely list.height in this case). So you can replace title.content.height with source in this case.
I notice that you're using a DoubleProperty type for expandedHeight in your example, but you need to keep looking at the value of the associated types. That's not very idiomatic. If you don't need this value to be reactive, a simple Double would suffice (but this would require that expandedHeight be declared as a var).
Combined, this produces the following code:
import scalafx.Includes._
import scalafx.application.JFXApp
import scalafx.beans.property.DoubleProperty
import scalafx.beans.value.ObservableValue
import scalafx.collections.ObservableBuffer
import scalafx.event.ActionEvent
import scalafx.scene.Scene
import scalafx.scene.control.cell.TextFieldListCell
import scalafx.scene.control.{Button, ListView, TitledPane}
import scalafx.scene.layout.BorderPane
object TitledPaneEndOfExpansion extends JFXApp {
var expandedHeight: Double = _
val data: ObservableBuffer[String] = new ObservableBuffer[String]() ++= List("some", "content", "for", "testing")
stage = new JFXApp.PrimaryStage {
title = "JavaFX: edit after rendering test"
val list: ListView[String] = new ListView[String](data) {
editable = true
cellFactory = TextFieldListCell.forListView()
height.onChange { (source: ObservableValue[Double, Number], oldValue: Number, newValue: Number) =>
expandedHeight = source.value
println("old height is: " + oldValue.doubleValue() + " new height is: " + newValue.doubleValue())
if (newValue.doubleValue() == expandedHeight) {
edit(1)
}
}
}
val titled: TitledPane = new TitledPane {
text = "titled"
content = list
}
scene = new Scene {
root = new BorderPane {
center = titled
bottom = new Button() {
text = "edit cell 1"
onAction = { _: ActionEvent => list.edit(1) }
}
}
}
expandedHeight = list.height.value //set to 400
list.edit(1)
}
}
Your code then compiles and runs.
Updated
ScalaFX is simply a wrapper for JavaFX: each JavaFX type has a corresponding ScalaFX type. ScalaFX provides implicit conversion functions to seamlessly convert, say, a JavaFX TitledPane to a ScalaFX TitledPane, and vice versa. However, there's no inheritance relationship between the two sets of objects. That is, a JavaFX TitledPane has no type relationship to a ScalaFX TitledPane. Casting between the two sets of objects is therefore a complicated process.
If you wanted to be able to cast titled.content correctly in order to access the height property of the contents more directly, you would need to get the property's value and explicitly pattern match on the result with the JavaFX version of the object, as follows:
import javafx.scene.control.{ListView => JFXListView}
import scalafx.Includes._
import scalafx.application.JFXApp
import scalafx.beans.property.DoubleProperty
import scalafx.beans.value.ObservableValue
import scalafx.collections.ObservableBuffer
import scalafx.event.ActionEvent
import scalafx.scene.Scene
import scalafx.scene.control.cell.TextFieldListCell
import scalafx.scene.control.{Button, ListView, TitledPane}
import scalafx.scene.layout.BorderPane
object TitledPaneEndOfExpansion extends JFXApp {
var expandedHeight: Double = _
val data: ObservableBuffer[String] = new ObservableBuffer[String]() ++= List("some", "content", "for", "testing")
stage = new JFXApp.PrimaryStage {
title = "JavaFX: edit after rendering test"
val list: ListView[String] = new ListView[String](data) {
editable = true
cellFactory = TextFieldListCell.forListView()
height.onChange { (source: ObservableValue[Double, Number], oldValue: Number, newValue: Number) =>
expandedHeight = titled.content.value match {
case lv: JFXListView[_] => lv.height.value
case _ => {
throw new RuntimeException(s"Unexpected content type: ${titled.content.getClass.getCanonicalName}")
}
}
println("old height is: " + oldValue.doubleValue() + " new height is: " + newValue.doubleValue())
if (newValue.doubleValue() == expandedHeight) {
edit(1)
}
}
}
val titled: TitledPane = new TitledPane {
text = "titled"
content = list
}
scene = new Scene {
root = new BorderPane {
center = titled
bottom = new Button() {
text = "edit cell 1"
onAction = { _: ActionEvent => list.edit(1) }
}
}
}
expandedHeight = titled.content.value match { //set to 400
case lv: JFXListView[_] => lv.height.value
case _ => throw new RuntimeException(s"Unexpected content type: ${titled.content.getClass.getCanonicalName}")
}
list.edit(1)
}
}
If you didn't have any other means of referencing the list object, that would be your only option.

Related

What's the easiest way to load single band Landsat 8 rasters from AWS and combine them into a single multiband RDD?

I'm using geotrellis to load geotiff rasters from Landsat 8 that are sitting on S3. However, they are stored on a per-band basis. I can use S3GeoTiff class to load individual bands, e.g.:
val options = S3GeoTiffRDD.Options(getS3Client = () => S3Client.ANONYMOUS)
val rddBlue = S3GeoTiffRDD.spatial("landsat-pds", "L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B2.TIF", options)
val rddGreen = S3GeoTiffRDD.spatial("landsat-pds", "L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B3.TIF", options)
val rddRed = S3GeoTiffRDD.spatial("landsat-pds", "L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B4.TIF", options)
But how do I go about combining them to generate an RGB raster, e.g.
val rddRGB = ??? // something like combineRDDs(rddRed, rddGreen, rddBlue)
From the RDDs of single files with no maxTileSize option set, you'll end up with RDDs of full images (one element). I'd recommend setting the maxTileSize option.
There's an overload that allows you to place extra information into the key. This is how I'd approach this problem generically.
Here is some code that does what you are looking for, which utilizes those options:
import geotrellis.raster._
import geotrellis.spark._
import geotrellis.spark.io.s3._
import geotrellis.vector._
import org.apache.spark.SparkContext
import org.apache.spark.rdd._
import java.net.URI
import scala.math.BigDecimal.RoundingMode
implicit val sc: SparkContext =
geotrellis.spark.util.SparkUtils.createLocalSparkContext("local[*]", "landsat-example")
try {
val options =
S3GeoTiffRDD.Options(
getS3Client = () => S3Client.ANONYMOUS,
maxTileSize = Some(512),
numPartitions = Some(100)
)
type LandsatKey = (ProjectedExtent, URI, Int)
// For each RDD, we're going to include more information in the key, including:
// - the ProjectedExtent
// - the URI
// - the future band value
def uriToKey(bandIndex: Int): (URI, ProjectedExtent) => LandsatKey =
{ (uri, pe) =>
(pe, uri, bandIndex)
}
// Read an RDD of source tiles for each of the bands.
val redSourceTiles =
S3GeoTiffRDD[ProjectedExtent, LandsatKey, Tile](
"landsat-pds",
"L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B2.TIF",
uriToKey(0),
options
)
val greenSourceTiles =
S3GeoTiffRDD[ProjectedExtent, LandsatKey, Tile](
"landsat-pds",
"L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B3.TIF",
uriToKey(1),
options
)
val blueSourceTiles =
S3GeoTiffRDD[ProjectedExtent, LandsatKey, Tile](
"landsat-pds",
"L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B4.TIF",
uriToKey(2),
options
)
// Union these together, rearrange the elements so that we'll be able to group by key,
// group them by key, and the rearrange again to produce multiband tiles.
val sourceTiles: RDD[(ProjectedExtent, MultibandTile)] = {
sc.union(redSourceTiles, greenSourceTiles, blueSourceTiles)
.map { case ((pe, uri, bandIndex), tile) =>
// Get the center of the tile, which we will join on
val (x, y) = (pe.extent.center.x, pe.extent.center.y)
// Round the center coordinates in case there's any floating point errors
val center =
(
BigDecimal(x).setScale(5, RoundingMode.HALF_UP).doubleValue(),
BigDecimal(y).setScale(5, RoundingMode.HALF_UP).doubleValue()
)
// Get the scene ID from the path
val sceneId = uri.getPath.split('/').reverse.drop(1).head
val newKey = (sceneId, center)
val newValue = (pe, bandIndex, tile)
(newKey, newValue)
}
.groupByKey()
.map { case (oldKey, groupedValues) =>
val projectedExtent = groupedValues.head._1
val bands = Array.ofDim[Tile](groupedValues.size)
for((_, bandIndex, tile) <- groupedValues) {
bands(bandIndex) = tile
}
(projectedExtent, MultibandTile(bands))
}
}
// From here, you could ingest the multiband layer.
// But for a simple test, we will rescale the bands and write them out to a single GeoTiff
import geotrellis.spark.tiling.FloatingLayoutScheme
import geotrellis.raster.io.geotiff.GeoTiff
val (_, metadata) = sourceTiles.collectMetadata[SpatialKey](FloatingLayoutScheme(512))
val tiles = sourceTiles.tileToLayout[SpatialKey](metadata)
val raster =
ContextRDD(tiles, metadata)
.withContext { rdd =>
rdd.mapValues { tile =>
// Magic numbers! These were created by fiddling around with
// numbers until some example landsat images looked good enough
// to put on a map for some other project.
val (min, max) = (4000, 15176)
def clamp(z: Int) = {
if(isData(z)) { if(z > max) { max } else if(z < min) { min } else { z } }
else { z }
}
val red = tile.band(0).map(clamp _).delayedConversion(ByteCellType).normalize(min, max, 0, 255)
val green = tile.band(1).map(clamp _).delayedConversion(ByteCellType).normalize(min, max, 0, 255)
val blue = tile.band(2).map(clamp _).delayedConversion(ByteCellType).normalize(min, max, 0, 255)
MultibandTile(red, green, blue)
}
}
.stitch
GeoTiff(raster, metadata.crs).write("/tmp/landsat-test.tif")
} finally {
sc.stop()
}

JSON.parse error on simplejson return in Django

I have a view page that currently has two columns of data shown, soon to be expanded to four. Each column contains the result of a QuerySet for that particular model.
Here's what I have in my views.py method:
if request.REQUEST["type"] == "text":
client = Client.objects.get(client_name = request.REQUEST["search"])
peerList = ClientPeers.objects.prefetch_related().filter(client = client.client)
compList = ClientCompetitors.objects.prefetch_related().filter(client = client.client)
else:
peerList = ClientPeers.objects.prefetch_related().filter(client = request.REQUEST["search"])
compList = ClientCompetitors.objects.prefetch_related().filter(client = request.REQUEST["search"])
for peer in peerList:
peerlst.append({"pid" : peer.parentorg.parentorg, "pname" : peer.parentorg.parentorgname})
for comp in compList:
complst.append({"cid" : comp.parentorg.parentorg, "cname" : comp.parentorg.parentorgname})
lst.append(simplejson.dumps(peerlst))
lst.append(simplejson.dumps(complst))
return HttpResponse(simplejson.dumps(lst), mimetype = "text/json")
This allows me to send a 2D array of data to the browser in the format
[ { //JSON }, { //JSON } ]
In my jQuery.ajax success function, I have
function handler(results) {
var data = JSON.parse(results);
for (var i = 0; i < data[0].length; i++)
$("#available_peers").append("<li>" + data[0][i].pname + "</li>");
for (var i = 0; i < data[1].length; i++)
$("#available_competitors").append("<li>" + data[1][i].cname + "</li>");
Firebug shows that the GET request works and I can see the data in the response tab. However, the console prints out
SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data
var data = JSON.parse(results)
This error disappears if I replace var data = JSON.parse(results) with
var peers = JSON.parse(data[0]);
var comps = JSON.parse(data[1]);
Why does one method work but another doesn't?
The jQuery ajax() call will make an intelligent guess as to the returned data type. In your example, function handler(results), the results variable will already be a decoded JSON object, containing two items in an array. The reason that JSON.parse(data[0]) works, is that you have returned JSON encoded data as a string.
Don't encode the individual list elements to JSON before placing in the output array:
lst.append(peerlst) # <-- Don't encode to JSON string here
lst.append(complst)
return HttpResponse(simplejson.dumps(lst), mimetype = "application/json") # <-- Single JSON encoding

docx4j create unnumbered / bullet list

I would like to create an unnumbered list with bullets using docx4j in my Word document. I have found the following code that is supposed to do the work. But whatever I try, the generated list is a numbered list! I use Word 2010, German version and docx4j-2.8.1.
wordMLPackage = WordprocessingMLPackage.createPackage();
ObjectFactory factory = new org.docx4j.wml.ObjectFactory();
P p = factory.createP();
org.docx4j.wml.Text t = factory.createText();
t.setValue(text);
org.docx4j.wml.R run = factory.createR();
run.getContent().add(t);
p.getContent().add(run);
org.docx4j.wml.PPr ppr = factory.createPPr();
p.setPPr(ppr);
// Create and add <w:numPr>
NumPr numPr = factory.createPPrBaseNumPr();
ppr.setNumPr(numPr);
// The <w:ilvl> element
Ilvl ilvlElement = factory.createPPrBaseNumPrIlvl();
numPr.setIlvl(ilvlElement);
ilvlElement.setVal(BigInteger.valueOf(0));
// The <w:numId> element
NumId numIdElement = factory.createPPrBaseNumPrNumId();
numPr.setNumId(numIdElement);
numIdElement.setVal(BigInteger.valueOf(1));
wordMLPackage.getMainDocumentPart().addObject(p);
Can someone help me to generate a real unordered, buletted list?!
Hope this helps you.
import org.docx4j.XmlUtils;
import org.docx4j.openpackaging.packages.WordprocessingMLPackage;
import org.docx4j.openpackaging.parts.WordprocessingML.NumberingDefinitionsPart;
import org.docx4j.wml.*;
import javax.xml.bind.JAXBException;
import java.io.File;
import java.math.BigInteger;
public class GenerateBulletedList {
private static final String BULLET_TEMPLATE ="<w:numbering xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">" +
"<w:abstractNum w:abstractNumId=\"0\">" +
"<w:nsid w:val=\"12D402B7\"/>" +
"<w:multiLevelType w:val=\"hybridMultilevel\"/>" +
"<w:tmpl w:val=\"AECAFC2E\"/>" +
"<w:lvl w:ilvl=\"0\" w:tplc=\"04090001\">" +
"<w:start w:val=\"1\"/>" +
"<w:numFmt w:val=\"bullet\"/>" +
"<w:lvlText w:val=\"\uF0B7\"/>" +
"<w:lvlJc w:val=\"left\"/>" +
"<w:pPr>" +
"<w:ind w:left=\"360\" w:hanging=\"360\"/>" +
"</w:pPr>" +
"<w:rPr>" +
"<w:rFonts w:ascii=\"Symbol\" w:hAnsi=\"Symbol\" w:hint=\"default\"/>" +
"</w:rPr>" +
"</w:lvl>" +
"</w:abstractNum>"+
"<w:num w:numId=\"1\">" +
"<w:abstractNumId w:val=\"0\"/>" +
"</w:num>" +
"</w:numbering>";
public static void main(String[] args) throws Exception{
WordprocessingMLPackage wordMLPackage = WordprocessingMLPackage.createPackage();
createBulletedList(wordMLPackage);
wordMLPackage.save(new File("Output.docx"));
}
private static void createBulletedList(WordprocessingMLPackage wordMLPackage)throws Exception{
NumberingDefinitionsPart ndp = new NumberingDefinitionsPart();
wordMLPackage.getMainDocumentPart().addTargetPart(ndp);
ndp.setJaxbElement((Numbering) XmlUtils.unmarshalString(BULLET_TEMPLATE));
wordMLPackage.getMainDocumentPart().addObject(createParagraph("India"));
wordMLPackage.getMainDocumentPart().addObject(createParagraph("United Kingdom"));
wordMLPackage.getMainDocumentPart().addObject(createParagraph("France"));
}
private static P createParagraph(String country) throws JAXBException {
ObjectFactory factory = new org.docx4j.wml.ObjectFactory();
P p = factory.createP();
String text =
"<w:r xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">" +
" <w:rPr>" +
"<w:b/>" +
" <w:rFonts w:ascii=\"Arial\" w:cs=\"Arial\"/><w:sz w:val=\"16\"/>" +
" </w:rPr>" +
"<w:t>" + country + "</w:t>" +
"</w:r>";
R r = (R) XmlUtils.unmarshalString(text);
org.docx4j.wml.R run = factory.createR();
run.getContent().add(r);
p.getContent().add(run);
org.docx4j.wml.PPr ppr = factory.createPPr();
p.setPPr(ppr);
// Create and add <w:numPr>
PPrBase.NumPr numPr = factory.createPPrBaseNumPr();
ppr.setNumPr(numPr);
// The <w:numId> element
PPrBase.NumPr.NumId numIdElement = factory.createPPrBaseNumPrNumId();
numPr.setNumId(numIdElement);
numIdElement.setVal(BigInteger.valueOf(1));
return p;
}
}
The code you have posted says "use list number 1, level 0".
Evidently that list is a numbered list.
Have a look in your numbering definitions part for a bulleted list, and use that one.
If you don't have a bulleted list there, you'll need to add it. You can upload a sample docx to the docx4j online demo, to have it generate appropriate content for you. Or see ListHelper for an example of how it can be done.
private static P getBulletedParagraph(Text text, int i) {
ObjectFactory objCreator = Context.getWmlObjectFactory(); // Object used to
create other Docx4j Objects
P paragraph = objCreator.createP(); // create Paragraph object
PPr ppr = objCreator.createPPr(); // create ppr
NumPr numpr = objCreator.createPPrBaseNumPr();
PStyle style = objCreator.createPPrBasePStyle();// create Pstyle
NumId numId = objCreator.createPPrBaseNumPrNumId();
numId.setVal(BigInteger.valueOf(6));
numpr.setNumId(numId);
R run = objCreator.createR();
Br br = objCreator.createBr();
run.getContent().add(text);
Ilvl iLevel = objCreator.createPPrBaseNumPrIlvl(); // create Ilvl Object
numpr.setIlvl(iLevel);
iLevel.setVal(BigInteger.valueOf(i)); // Set ilvl value
ppr.setNumPr(numpr);
style.setVal("ListParagraph"); // set value to ListParagraph
ppr.setPStyle(style);
paragraph.setPPr(ppr);
paragraph.getContent().add(run);
// paragraph.getContent().add(br); Adds line breaks
return paragraph;
}
I believe this is what you're looking for. This method will return you a paragraph object that has bullets. If you uncomment out the last line before returning paragraph, your paragraph object will also contain line breaks. If you didn't know, "Ilvl", or "eye-level" means indentation. Its the same as clicking the tab button when typing to a document the conventional way. Setting the ilvl element to 1 is the same as clicking tab 1 time. Setting it to 2 is the same as clicking the tab button 2 times, and so on. So it doesn't matter what number you give it. Although it will change the type of bullet you get. What really matters is the numid. Setting the numid to 6 will give you bullets instead of numbers. You will also need to set the PPr style to "ListParagraph". I hope this helps.

Get Folder size of HDFS from java

I have to HDFS folder size which is having sub directories from java.
From command line we can use -dus option, But anyone can help me on how to get the same using java.
The getSpaceConsumed() function in the ContentSummary class will return the actual space the file/directory occupies in the cluster i.e. it takes into account the replication factor set for the cluster.
For instance, if the replication factor in the hadoop cluster is set to 3 and the directory size is 1.5GB, the getSpaceConsumed() function will return the value as 4.5GB.
Using getLength() function in the ContentSummary class will return you the actual file/directory size.
You could use getContentSummary(Path f) method provided by the class FileSystem. It returns a ContentSummary object on which the getSpaceConsumed() method can be called which will give you the size of directory in bytes.
Usage :
package org.myorg.hdfsdemo;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class GetDirSize {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
Configuration config = new Configuration();
config.addResource(new Path(
"/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));
config.addResource(new Path(
"/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));
FileSystem fs = FileSystem.get(config);
Path filenamePath = new Path("/inputdir");
System.out.println("SIZE OF THE HDFS DIRECTORY : " + fs.getContentSummary(filenamePath).getSpaceConsumed());
}
}
HTH
Thank you guys.
Scala version
package com.beloblotskiy.hdfsstats.model.hdfs
import java.nio.file.{Files => NioFiles, Paths => NioPaths}
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.FileSystem
import org.apache.hadoop.fs.Path
import org.apache.commons.io.IOUtils
import java.nio.file.{Files => NioFiles}
import java.nio.file.{Paths => NioPaths}
import com.beloblotskiy.hdfsstats.common.Settings
/**
* HDFS utilities
* #author v-abelablotski
*/
object HdfsOps {
private val conf = new Configuration()
conf.addResource(new Path(Settings.pathToCoreSiteXml))
conf.addResource(new Path(Settings.pathToHdfsSiteXml))
private val fs = FileSystem.get(conf)
/**
* Calculates disk usage with replication factor.
* If function returns 3G for folder with replication factor = 3, it means HDFS has 1G total files size multiplied by 3 copies space usage.
*/
def duWithReplication(path: String): Long = {
val fsPath = new Path(path);
fs.getContentSummary(fsPath).getSpaceConsumed()
}
/**
* Calculates disk usage without pay attention to replication factor.
* Result will be the same with hadopp fs -du /hdfs/path/to/directory
*/
def du(path: String): Long = {
val fsPath = new Path(path);
fs.getContentSummary(fsPath).getLength()
}
//...
}
Spark-shell tool to show all tables and their consumption
A typical and illustrative tool for Spark-shell, looping loop over all bases, tables and partitions, to get sizes and report into a CSV file:
// sshell -i script.scala > ls.csv
import org.apache.hadoop.fs.{FileSystem, Path}
def cutPath (thePath: String, toCut: Boolean = true) : String =
if (toCut) thePath.replaceAll("^.+/", "") else thePath
val warehouse = "/apps/hive/warehouse" // the Hive default location for all databases
val fs = FileSystem.get( sc.hadoopConfiguration )
println(s"base,table,partitions,bytes")
fs.listStatus( new Path(warehouse) ).foreach( x => {
val b = x.getPath.toString
fs.listStatus( new Path(b) ).foreach( x => {
val t = x.getPath.toString
var parts = 0; var size = 0L; // var size3 = 0L
fs.listStatus( new Path(t) ).foreach( x => {
// partition path is x.getPath.toString
val p_cont = fs.getContentSummary(x.getPath)
parts = parts + 1
size = size + p_cont.getLength
//size3 = size3 + p_cont.getSpaceConsumed
}) // t loop
println(s"${cutPath(b)},${cutPath(t)},${parts},${size}")
// display opt org.apache.commons.io.FileUtils.byteCountToDisplaySize(size)
}) // b loop
}) // warehouse loop
System.exit(0) // get out from spark-shell
PS: I checked, size3 is always 3*size, no extra information.

Restrict TextField to act like a numeric stepper

I am making a numeric stepper from scratch, so I want my text field to only accept numbers in this format: xx.x, x.x, x, or xx where x is a number. For example:
Acceptable numbers:
1
22
15.5
3.5
None Acceptable numbers:
213
33.15
4332
1.65
Maybe this will help some how:
http://livedocs.adobe.com/flash/9.0/ActionScriptLangRefV3/flash/text/TextField.html#restrict
This is what I got so far:
var tx:TextField = new TextField();
tx.restrict="0-9."; //Maybe there is a regular expression string for this?
tx.type=TextFieldType.INPUT;
tx.border=true;
You can copy past this in flash and it should work.
Thank you very much for your help good sirs.
Very similar to TheDarklins answer, but a little more elegant. And actually renders _tf.restrict obsolete, but I would still recommend using it.
_tf.addEventListener(TextEvent.TEXT_INPUT, _onTextInput_validate);
Both of these event listeners here do the EXACT same function identically. One is written in a one line for those who like smaller code. The other is for those who like to see what's going on line by line.
private function _onTextInput_validate(__e:TextEvent):void
{
if ( !/^\d{1,2}(?:\.(?:\d)?)?$/.test(TextField(__e.currentTarget).text.substring(0, TextField(__e.currentTarget).selectionBeginIndex) + __e.text + TextField(__e.currentTarget).text.substring(TextField(__e.currentTarget).selectionEndIndex)) ) __e.preventDefault();
}
for a more broken down version of the event listener
private function _onTextInput_validate(__e:TextEvent):void
{
var __reg:RegExp;
var __tf:TextField;
var __text:String;
// set the textfield thats causing the event.
__tf = TextField(__e.currentTarget);
// Set the regular expression.
__reg = new RegExp("\\d{1,2}(?:\\.(?:\\d)?)?$");
// or depending on how you like to write it.
__reg = /^\d{1,2}(?:\.(?:\d)?)?$/;
// Set all text before the selection.
__text = __tf.text.substring(0, __tf.selectionBeginIndex);
// Set the text entered.
__text += __e.text;
// Set the text After the selection, since the entered text will replace any selected text that may be entered
__text += __tf.text.substring(__tf.selectionEndIndex);
// If test fails, prevent default
if ( !__reg.test(__text) )
{
__e.preventDefault();
}
}
I have had to allow xx. as a valid response otherwise you would need to type 123 then go back a space and type . for 12.3. That is JUST NOT NICE. So 12. is now technically valid.
package
{
import flash.display.Sprite;
import flash.text.TextField;
import flash.text.TextFieldType;
import flash.events.TextEvent;
public class DecimalPlaces extends Sprite
{
public function DecimalPlaces()
{
var tf:TextField = new TextField();
tf.type = TextFieldType.INPUT;
tf.border = true;
tf.width = 200;
tf.height = 16;
tf.x = tf.y = 20;
tf.restrict = ".0-9"
tf.addEventListener(TextEvent.TEXT_INPUT, restrictDecimalPlaces);
addChild(tf);
}
function restrictDecimalPlaces(evt:TextEvent):void
{
var matches:Array = evt.currentTarget.text.match(/\./g);
var allowedDecimalPlaces:uint = 1;
if ((evt.text == "." && matches.length >= 1) ||
(matches.length == 1 && (evt.currentTarget.text.lastIndexOf(".") + allowedDecimalPlaces < evt.currentTarget.text.length)))
evt.preventDefault();
}
}
}