Question: Can I import data from other applications into GoldenGem? What are the
data file conventions?

Question: How can I load stocks from the London Stock exchange like BAT?

>Answer:  Append the symbol .L  to the ticker, so you will list it in the list of
>related tickers as BAT.L


Question: 
If I spend several hours training a data set can I save it for next
 time, and go on to my next data set training, test?

>Answer: Unfortunately, no there is no way currently to save 
>the constants. It is only a few kb of data so it would 
>be possible to add that feature. Some neural networks 
>are pre-trained, as you may know. 
>
>However, You can save the  input data, If you have loaded 
>it from the internet, go to file: browse for new file and
> right click and rename Internet.txt (or just Internet if your computer
>hides file extensions) to a new name. If you have imported
>from a folder, go to Combined.txt and rename it.
>
>Then to recover the fit it is necessary to re-run on 
>the input data. 


Question: 
Very tiny prediction line (after the blue vertical line) any way to 
magnify that? 


> Answer: I like the idea of magnifying the prediction line. I am sort of worried 
> it might confuse people though, as it would be a different scale than 
> the rest of the graph. Therefore I have recently added a magnifier for that
> purpose.


Question: 
I am a little confused on the number of stocks to use. How bad is it 
if I just use one? Under the click on "recommended settings" its says a 
fit is more likely to be real if fewer tickers are used?




> Answer: The question about how many stocks is important. If you use too many,
> it is just too easy for the net to find a connection
> for essentially random reasons.

> With 1 stock only, you would only see if patterns relating volume and 
> price have historically affected the price. That is something, but 
> something you could do yourself by looking at the price and volume graphs. 
> Or, I imagine Trademan can do that. Someone named Luke Bairan has written 
> to me that high price and high volume means the price will go up more, 
> and low price and high volume means it will go down more. Certainly 
> a nerual network or a human being could work that out equally well. 
>
>
>  Some articles say 20 inputs (which would be 10 shares) is ideal, 
>  but others have gotten a match which wasn't real that way. It is likely five shares 
> (which is 10 inputs) is about ideal. 

Question: Can this program input indices, such as ^DJI (the Dow Jones Industrial)?

> Answer: That feature has recently been added. If you update your copy of GoldenGem from
> the local site it will be there. Currently indices that are in Yahoo with ^  or in MSN with $
> appear to be supported.

Question:  I would like to use as inputs the high and low price of a share, as a measure
of volatility. How would I do that?

>Answer:   Unless you decide to change the settings, GoldenGem looks at Close and Volume (positions 6 and 7). 
>The available columns in a data file coming from StockDownloader are Date Ticker, Open, High, Low, Close, Volume. 
>
>If you want the program to look at High and Low instead of Close and Volume  you would replace the 
>file position numbers which are currently 2,1,6,7  with  2,1,4,5.   then it will use both
>high and low, and the graph will show the 'high' price if you display it. If you use 1,2,5,4 the results will
>be the same, but the graph (and prediction) show the low price for the day instead of high.

Question:
I get an error message saying none of the tickers are in the selected file. But
when I go to File: View Loaded Data, they are there.

>Answer: Check the position in the text field is correct for the file position number (above where
>it says Tick). If you see tickers in column 2 make sure the tick number is set to 2.
>Also, check that the file position number field doesn't have 
>spurious text you may have accidentally typed in. You can view the whole ticker position
>field using the right and left arrow keys. The field is something like ten characters
>wide.


Question:  How do I get a one day prediction?

>Answer: If you set the 'days' slider to 1 you would get a single day's prediction. 


Question: Is it possible for the neural network to analyze several factors at once (i.e. 
crude oil prices, gdp, interest rates) to better forecast the movement of the DOW or S&P 500?


>Answer: Yes, it is a good idea to analyze several factors at once. It is possible to enter various 
>indicators such as ^DJI for the dow jones index etc. Also I think interest rate tickers work.
>I have tried using interest rates myself but I found that since they change so slowly, there is 
>no useful activity, I just couldn't see how the neural net could learn much due to the lack of variation.

>However, if you have data that might credibly affect the stock you want to analyze, it may come up already 
>as a ticker in StockDownloader.



Question: Is there any way to graph the volume information?


>Answer: That feature has recently been added.


Question:  The StockDownloader says 'no connection' even when I am connected to the internet.

>Answer: Check if you are using a direct connection to the internet. GoldenGem does not know
>how to pass a password and user name to a proxy connection, and this will result in
>a 'no connection' error.  Another possibility: if you have uninstalled Internet Explorer you may have
>uninstalled some necessary system components, even if you use another browser like
>FireFox. Another possibility may be that one of the data providers has changed its conventions. 
>We try to keep up as best we can with current data providers. 


Question:  How do I find a list of tickers that are likely to be mathematically related, over time?  
I've read the wikipedia link and it didn't help me on this point.   Can you please give me a working 
example and tell me how many tickers might be needed for the software to work?


>Answer: For some time, the tickers that come up the first time the program is used were working, which are 
>AMD, CSCO, INTC  These no longer predict well but if QQQQ is added it does do a bit better. 
>But it is most important for you yourself to know that there is a logical and causal relationship. 
>This group of shares was suggested by a member of a newsgroup. The correct answer to your question is that 
>it is an investment question and not a mathematical one so I should really not comment.


Question: Once I've downloaded a list of tickers, how many neural network levels should be entered?  
There's no explanation of this on your website.


>Answer:The default number comes up automatically which is based on a standard formula from a research paper. 
>It is universally agreed that 3 levels is best. The number of neurons in all but the middle level is 
>then fixed, and you can try different values for the level 2 neurons. You also can find a lot of opinion 
>by searching Google about it. As I say, the choice that comes up is a more or less standard agreed choice but it is 
>not completely agreed.



Question (re the autotrain version): Once I've chosen a stock from the drop down menu and clicked auto-analyse, 
it seems to go on forever  analysing thousands of iterations.....how long until completion in most cases?  
Is there a way to limit this?


>Answer:Try pressing Reset before you press the autotrain button, if you have tried training manually. This will 
>tell it you want it to start from the beginning. If the leftmost light never turns yellow (which is rare) it is 
>just a very bad choice of shares. If the leftmost light starts going yellow it should either advise you to 
>continue waiting or tell you to give up (the words appear in grey at the top of the white screen).
 


Question: Say I choose ticker BHP.AX from the menu and analyse it.  Does this mean that the software 
is trying to predict future price movements of BHP.AX by reading the other tickers on the list of 
related stocks?  Does it use all of them or some of them?


>Answer: It uses all of them. That is important because it is actually a reason to keep the ticker list to a
>minimum number of shares, and to try not only adding to the list but also deleting from the list.
>The more shares you include (even unrelated ones) the better  the backtesting can be and this
>is not always good.  This is for an obvious  reason that there will be more and more coincidental  
>ways of making patterns. It uses them all. So it is best to exclude from the list of shares things known to be 
>irrelevant. Including extra irrelevant shares can give misleadingly good backtesting which would not stand up to a 
>validation set. Whereas if both lights go green with a small list of shares (three or four) it is more likely
>to be an actual causal relationship that is found.



Question:  How do I ensure that both lights go green - is there some way of increasing the possibility of this?  
I'm guessing that it comes down to picking one's ticker list properly??


>Yes it does come down to that.   Exactly right.

>Moreover it should really be more than a ticker list. You really should try to find data such as 
>commodity prices etc that are directly related to the target share price. Otherwise it is not 
>much more than a game. If you know of online data sources directly related to particular shares let us know, 
>we'll make sure the program can access them. But most sources of data that are very specific are not 
>publicly available online.

Question: I find the software quite useful in predicting the Dow.  It would 
be helpful if you could explain more about finding a mathematical relationship 
between tickers.


 
>If you're getting some success predicting the Dow, then you don't need any advice about 
>finding a mathematical relationship!  I usually tell people not to try to predict indices, 
>but to use indices along with representative shares which may  have a causal relation, to 
>predict a particular share. If you think about it, predicting the Dow, an average, is 
>equivalent to predicting each  constituent share with better than 50% accuracy. It could 
>be possible, but would be  quite an achievement. Are you sure it is working? Instead of 
>me giving you advice, why  not keep me posted on your long term success/failure 
>at predicting the Dow?



>The answer to how to find a mathematical relationship would be to look not into Mathematics 
>but into people who write about the structure and relations in the market. Also to use 
>mathematical relationships which everyone knows about, and which are obvious, but aren't 
>usually used in combination. Along the lines that exchange rates between three countries A:B  and B:C  
>affect the share price of a manufacturer in country B who buys raw materials from country A to sell a 
>product in country C, etc.


Question: What are "columns"?.  How are they measured? What are the numerical values actually?  
Are they the current stock price or the percent change?  I'm guessing that there is a 
conflict between file types (Type Mismatch Error) in that the text file must be comma 
delimited csv files not plain text.  Do I do this on a spreadsheet with xls files?  
I'm using Windows 2000 SP4 on NTFS. In short,  the data files contain two values.  
The symbol and the value of something about the symbol.   How do I enter these into the program text files?



>Answer: If you want a single data file with more than one symbol, it is best to use .txt files 
>instead of .csv files.

>The first thing to do is make sure in the control panel, in Folder Options, you have 
>unchecked "hide file extensions for known file types" because you want to be able to see 
>file extensions.

>Then just to be sure you have a plain text file, open it with Notepad, and make sure 
>it is readable there. Make sure the filename ends in .txt  such as  mydata.txt

>If it is not readable in Notepad, make sure your application exports it as  .txt or if 
>worse comes to worse, copy and paste it into Notepad and then save it.

>Wordpad has a greater data capacity, but has the annoying habit of trying to save 
>files as .rtf files instead of .txt. If you use Wordpad instead of Notepad make sure 
>you choose save as plain text file in the file saving window.

>All I mean by columns is the obvious thing. It sounds like you have symbols in the 
>first column and data in the second like

   IBM, 34
   MSFT, 45
   IBM, 46

>etcetera.

>The way the separate symbols are ordered does not matter, so the following are equivalent

   IBM, 34
   MSFT, 50
   IBM, 33
   MSFT, 51

>is the same as

   IBM, 34
   IBM, 33
   MSFT, 50
   MSFT, 51

>But it DOES matter that we do not switch the MSFT 50  and MSFT 51  for example as the 
>first one is supposed to refer to an earlier data point (business day most likely 
>if your data is daily).

>Now, it also is important that in such a file, the date or time of the IBM, 34 must 
>be the same as the date or time of the MSFT, 50. So either of the two data files 
>shown above represent two days of data, the first day having IBM at 34 and MSFT at 50, 
>the second day having IBM at 33, MSFT at 51.

>Now, for such a file you want to have the file position numbers

   date     tick     close     vol
  (blank)   1          2       (blank)

>or you could put the number of a nonexistent column like 3 in for the blank.

>This is saying the ticker names are in the first entry of each row,
>and the data in the second entry.

>The tickers in the window can be a subset of the symbols actually in the
>file, so you can have a very large file with lots and lots of data, and
>choose various ticker symbols to investigate as you go along.

>If your file is mydata.txt then go to the FILE: menu and choose "Browse for new file"
>and click your file.

>Now it will load whatever tickers you have listed. If you want to load
>a new subset of tickers, you can just change the ticker names
>in the window and choose now in the FILE: menu the first option, this
>will be to reload those new tickers from the existing file.




Question: What are the numerical values actually?  Are they the current stock 
price or the percent change?


>Answer: The numbers can be anything at all, but you should be aware that they are 
>normalized to mean value zero and standard deviation one before being input.

>The usual decision is to enter the actual values, but recently an investor told me 
>he was interested in having as variables the difference between current price and 
>price 30 days ago, and also the difference between current price and price in 
>January 08, as well as the current price.

>For that situation it would be a mistake to enter separately the current price, 
>the price 30 days ago and the price on a day in January 08. The last of these is a 
>constant, and as a separate variable would be normalized to mean zero, which would 
>imply the input would just be zero.

>So there are rare circumstances when differences do want to be used as inputs, but 
>generally it is not differences but actual values that should be the inputs.