computoman

One thing I like to do is page scraping. I can let the computer pull the data I want for me. Unfortunately, if the page is changed, I have to change the code. This happened recently.  Since my code is generic enough, the changes were no issue. There are three scripts. One for love, one for daily, and one for monthly. When I updated the scripts, made them a bit more generic. Usage: $ ./horoscope.sh astrologysign Save to file" $ ./horoscope.sh astrologysign > savefilename Daily Taurus Daily Thursday 3rd December 2015 Your opinions and ideas could be central in a work discussion. The only 'problem' here is that you might seem to be changing your mind. It's likely that you're not happy with a proposal. Perhaps you feel that the other party hasn't given due thought to commercial realities. At a quite different level you could give thought once more to buying a large item. --------------------------------------------- [code]  #########...

Ever wonder what would be like to have a local web app for the time without having to connect to some far off unknown place with who knows what hidden features. Now you can have local clock time on your server   <code> <table border = 2 > < tr > < td > < script > var mydate=new Date() var year=mydate.getYear() if (year < 1000) year+=1900 var day=mydate.getDay() var month=mydate.getMonth() var daym=mydate.getDate() if (daym<10) daym="0"+daym var dayarray=new Array("Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday") var montharray=new Array("January","February","March","April","May","June","July","August","September","October","November","December") document.write("<small><font color='000000' face=...

When I first made the following cartoon it was pretty much tongue in cheek. But really when you think about it, it is a blessing in disguise. What if you could localize a lot of the web you use a lot without having to use any bandwidth. Actually we already do it with our local web server now that I think about it. If you were a traveling chef you could go choose a food menu. and from that menu, you could do several things. Such as use several calculators. Such a a food conversion chart. Or even have your recipe book with you. And the neatest of all things you could have a virtual site from the same server to show your clients what they could have for dinner. Is that not all awesome! All from that little server without having to go to the internet.

If you go to a web page, you get to see all the pretty pictures and advertisements. What if you not only just want part of the page and to avoid all the advertisements or you do not have a gui based terminal? You can accomplish that with what is known as a page scrape. A page scrape extracts just the data we want without even having to look at a gui web page. What really happens we sort of print only the text from a page to the screen. Take the following page: $ firefox http://www.creators.com/lifestylefeatures/horoscopes/holiday-mathis-weekly.html But all we really want is this part of the page. Now consider we want to do everything from the command line, so that all you can use is text characters. No problem, but you have to use the mouse to highlight what we need and then paste it to the screen or to a file. The same page as text where we have scrolled down to where the data we want is.  $ lynx "http://www.creators.com/lifestylefeatures/horoscopes/holiday-mathis-weekly.html...

If you go to a web page, you get to see all the pretty pictures and advertisements. What if you not only just want part of the page and to avoid all the advertisements or you do not have a gui based terminal? You can accomplish that with what is known as a page scrape. A page scrape extracts just the data we want without even having to look at a gui web page. What really happens we sort of print only the text from a page to the screen. Take the following page: $ firefox http://www.creators.com/lifestylefeatures/horoscopes/holiday-mathis-weekly.html But all we really want is this part of the page. Now consider we want to do everything from the command line, so that all you can use is text characters. No problem, but you have to use the mouse to highlight what we need and then paste it to the screen or to a file. The same page as text where we have scrolled down to where the data we want is.  $ lynx "http://www.creators.com/lifestylefeatures/horoscopes/holiday-m...

Legacy way to combine text and a graphic. <table>     <tr>         <td>                 <img src="prof.png" alt="">        </td>        <td>06/01/2008 The Info pages are done <br> 06/09/2008 Locally on-line as of 03/05/2008 <br> 06/08/2009 Some web pages updated <br> 06/30/2010 Redoing student services web pages. <br> 09/15/2014 Menus are restructured        </td>    </tr> </table ---------------------------------- Change text in many files with one command. Say you want to duplicate some web pages to another server, but the web server names are different. That means all the links in the pages will be wrong. For example take the following simple web page: [code] <html> <t...

Decided to take my score scraping script and apply it to other media. Per se maybe we just want the headlines of CNN.com. Used essentially the same set of instructions on a local news site with results. You will want to use the same rough logic or pseudocode: Dump web page as ascii (text) to a disk file. Read disk file one line at a time in, but ignore all lines till you get what is needed. Now continue reading one line at a time, but ignore certain lines With the line you read in, output what is needed while editing out unwanted characters. Stop output when you get to a point nothing else is needed. Finnish reading file one line at a time. End. You might get results like this ignoring equals signs. ========= THE LATEST * Source: Joan Rivers' doc took selfie * Freak accident kills hero bus driver * IOS 8 is live: How to get it * Five iOS 8 features you'll love * NEW Billionaire tells big named computer company: Innovate * Obama stands ...

Using bash over simplified to create a web page so that you do not have to go to a bunch of web pages and therefore you miss the ads. Update: You can find the similar code for the sub-programs in earlier articles. Update new section to add to script #-------------------------------------------------------- # cnn echo "<h3>CNN Headines</h3>" >> report.html echo "<pre>" >> report.html # creates cnn.txt ./cnn.sh > cnn.txt echo "<pre>" >> report.html cat cnn.txt >> report.html echo "</pre>" >> report.html Original script #---------------------------------------------- # daily journal echo "<html>" > report.html echo "<body>" >> report.html echo "<script>" >> report.html echo "var d=new Date();" >> report.html echo "document.write('<h2> Journal for ');" >> repo...

One thing that is good for an network administrator is to be able to see if a web server is up without having to load a browser. Of course the web browsers usually run on port 80. Here is a short script to do that. Grab a root webpage and save the code. Great for seeing if a server is up. Usage:  ./webpageget.sh site outputfile ./getwebpage.sh oesrvr1 oesrvr1_rootpage rm: cannot remove `oesrvr1_rootpage’: No such file or directory <html> <head> <title> Offshore Educators (Main Page) 1 </title> <base href=” http://oesrvr1/oe/”&gt ; </head> <FRAMESET ROWS=”15%,*”> <FRAME SRC=”title.html” NAME=TITLE SCROLLING=NO> <FRAMESET COLS=”20%,*” FRAMEBORDER=no> <FRAME SRC=”tools.html” NAME=SIDEBAR noresize> <FRAME SRC=”notes.html” NAME=NOTES noresize> </FRAMESET> <NOFRAMES> <H1> Offshore Educators</H1> No frames? No Problem! Take a look at our <A HREF=”plain.html”>no-frames...

Ever wished you could get data from a web page without reading the whole web page, or for even just getting data from a server.   One thing I like to do is get the football scores. Find that SI.com and some other sites have so much going on it takes forever to navigate the pages. Not only that, you are subjected to all the ads. Just give me the scores so I can move on. Data mining allows me to do that.  In other words, the computer can be your personal secretary to get all the data you need for your special reports. without you having to do all that hard work and the extra time to be spent. Written a series of beginner guides for data mining. You can find them at. http://www.instructables.com/id/Data-mining/ Note:  The football score capturing script works best for the preceding week or earlier in the same season. Update: Let's take what we have learned already and apply it. Showed you how to extract data to make your own web page and also showed y...

Say you want to duplicate some web pages to another server, but the web server names are different. That means all the links in the pages will be wrong. For example take the following simple web page: [code] <html> <title> Offshore Educators (C1001) </title>  <title> Send us some comments </title>  <body bgcolor="aquamarine">     <CENTER><h2><B><I>Offshore Educators</I></B></h2></CENTER> Sylabus: C1001 - Introduction to computing 1. <hr> <br> <p> This course introduces students to fundamental hardware concepts in the use of microcomputers and to some practical applications of software. The course focuses primarily on word processing, spreadsheets and business charts and presentation software using the Open Office products on the Linux platform. The course examines the use of computers for communication and collaboration including e-mail, conferencing...

As a tech, not so long ago when installing MSWindows on users computers at a former job, people were dismayed at the fact of having to use a mouse via a gui (graphical user interface) with wysiwyg (what you see is what you get pronounced wizzywig). They protested they preferred the keyboard only environment. They exclaimed the keyboard environment was more efficient and exacting. In fact, we who were installing the new environment were verbally accosted for the change over. Now if you suggest using a system that does not have a gui, you go through the same negative feedback. Decided to do a new kind of roughing it by building a system that was technically obsolete using a non-gui environment, but the operating system was up to date. Yep, that means everything has to be done through the keyboard.  Makes me think of the "Star Trek" movie where the Scotty character was using an old Apple Macintosh computer and could not get it to work. He was trying to talk to the mouse. He w...

Say you just need one bit of information off the web such as the latest football score. That means you have to start your browser, use a bookmark (or manually type in the url), and then finally scan the page for the information you need. In another case, you are logged into a server via the command line where no web browser is available. Page scraping is the answer to your needs. This may not be the best example, but say you want your horoscope for the day. All you need to do is to fire up a terminal if you are not already in a terminal and just type a few letters and boom there it is. Here is the page we want to get the data from: You can use just a small bit of program code to solve the problem (Use your favorite editor to create the program and the datafile and then make ghp executable with "chmod +x ghp"). ghp: [code] # Get today's horoscope echo "--------------------------------------------" # character width cw=60 hsign=$1 hsign="`echo $h...